From greg.ewing at canterbury.ac.nz Thu Jun 1 03:58:45 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 01 Jun 2017 19:58:45 +1200 Subject: How to install Python package from source on Windows In-Reply-To: References: <004701d2d979$0e2c2510$2860e747@sambora> Message-ID: Deborah Swanson wrote: > I got one suggestion that I could just copy the files to "the appropriate > directories", That was me, but I've just had a look at the source, and it seems the core functionality is implemented in C, so scrub that idea. But all is not lost. I've found this: https://pypi.python.org/pypi/namedlist "Similar to namedtuple, but instances are mutable." It's all pure Python -- I checked this time! -- so you shouldn't have any trouble installing it. In fact it's just a single source file, namedlist.py. Put that in your site-packages folder and you should be good to go. -- Greg From greg.ewing at canterbury.ac.nz Thu Jun 1 04:12:24 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 01 Jun 2017 20:12:24 +1200 Subject: How to install Python package from source on Windows In-Reply-To: References: <004701d2d979$0e2c2510$2860e747@sambora> Message-ID: Deborah Swanson wrote: > Why do you care so deeply what pip does on an operating system that is > no longer supported? Being pipless is a sufficiently distressing fate that we'd like to help, even if you didn't explicitly request it. Particularly since it seems like such an unnecessary one, because installing pip shouldn't require any kind of compiler on any version of any operating system, supported or otherwise. So the fact that it apparently does on your system is very puzzling, and some of us would like to solve that puzzle. -- Greg From greg.ewing at canterbury.ac.nz Thu Jun 1 04:15:12 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 01 Jun 2017 20:15:12 +1200 Subject: How to install Python package from source on Windows In-Reply-To: References: <005601d2d983$2593ebc0$2860e747@sambora> Message-ID: Deborah Swanson wrote: > I have already offered to do whatever you would like me to do on this > system If you wouldn't mind, I'd like to see the output from this command: python -m ensurepip -- Greg From steve+python at pearwood.info Thu Jun 1 05:11:00 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 01 Jun 2017 19:11:00 +1000 Subject: Working with dictionaries and keys help please! References: <53455047-7b3a-4874-ab3d-ed02957b5c97@googlegroups.com> Message-ID: <592fda25$0$1586$c3e8da3$5496439d@news.astraweb.com> On Thu, 1 Jun 2017 10:29 am, David D wrote: > I have a dictionary with a 10 people, the key being a number (0-10) and the > value being the people's name. I am in the processing of Insert, Adding and > deleting from the dictionary. All seems well until I delete a person and add > a new one. The numbers (keys) do not change and so I am getting an > non-sequential set of numbers for the keys. Is there a way of performing this > where the key will update so that is continues to work sequentially? Here is > what I mean > > Print dictionary > {0 John, 1 David, 2 Phil, 3 Bob} > > remove 1 David > {0 John, 2 Phil, 3 Bob} > > How can I get it so that the VALUE will reset and it will look like this after > both adding or deleting? > > {0 John, 1 Phil, 2 Bob} You can't. That's not how dicts work. The dictionary key is similar to the primary key in a database. The fact that they start off as sequential values is irrelevant. Imagine that you create an account for Phil, and he gets account number 2; then Bob gets account number 3: print(accounts) => {0: John, 1: David, 2: Phil, 3: Bob} Then you delete David's account. Then Phil calls you, tells you his account number is 2, and you access Bob's records instead of Phil's! Database keys should never be changed or reused. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From greg.ewing at canterbury.ac.nz Thu Jun 1 06:05:51 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 01 Jun 2017 22:05:51 +1200 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: <3dvqic92ts7tei230l03hnhr1b51794fs2@4ax.com> <4cdric1iau91j5p32mecn407ta8a0ad8g4@4ax.com> Message-ID: Chris Angelico wrote: > In the DB API 3.0, what I would like to see is that the connection > becomes a context manager that gives you a transaction object. With > that transaction, you can perform queries, and you could create > cursors from it if you want them, or just use a single in-built > cursor. But a cursor would relate to a result set, and would have no > meaning outside of it. A while back I wrote a dedicated wrapper around the Firebird API (I wasn't impressed by the DB API, for many of the reasons being discussed here), and this is almost exactly how I structured it. Except I didn't really have cursors as an explicit concept at all -- executing a query just gave you an object that you iterate over to get the results. It just seemed the most Pythonic way to do it. It helped that the Firebird API itself was structured that way, so all I really had to do is wrap it in the most straightforward way. Is there any serious work being done on a DB API 3.0? If there is, I'd be interested in helping with the design. -- Greg From jon+usenet at unequivocal.eu Thu Jun 1 06:12:31 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Thu, 1 Jun 2017 10:12:31 -0000 (UTC) Subject: Python DB API - commit() v. execute("commit transaction")? References: <3dvqic92ts7tei230l03hnhr1b51794fs2@4ax.com> <4cdric1iau91j5p32mecn407ta8a0ad8g4@4ax.com> Message-ID: On 2017-06-01, Gregory Ewing wrote: > Is there any serious work being done on a DB API 3.0? > If there is, I'd be interested in helping with the design. There are a bunch of existing APIs in other languages that can easily be copied ;-) The good news is of course that since the DB-API 'Connection' class has so few existing methods, adding new ones while maintaining backwards-compatibility should be easy. From greg.ewing at canterbury.ac.nz Thu Jun 1 06:14:52 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 01 Jun 2017 22:14:52 +1200 Subject: Circular iteration on tuple starting from a specific index In-Reply-To: References: Message-ID: guillaume.paulet at giome.fr wrote: > def cycle_once(iterable, start): > return chain(islice(iterable, start, None), islice(iterable, start)) Another variation, maybe slightly more efficient: from itertools import islice, cycle def cycle_once(iterable, start): return islice(cycle(iterable), start, start + len(iterable)) -- Greg From rosuav at gmail.com Thu Jun 1 06:26:28 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Jun 2017 20:26:28 +1000 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: <3dvqic92ts7tei230l03hnhr1b51794fs2@4ax.com> <4cdric1iau91j5p32mecn407ta8a0ad8g4@4ax.com> Message-ID: On Thu, Jun 1, 2017 at 8:05 PM, Gregory Ewing wrote: > A while back I wrote a dedicated wrapper around the Firebird > API (I wasn't impressed by the DB API, for many of the reasons > being discussed here), and this is almost exactly how I structured > it. Except I didn't really have cursors as an explicit concept at > all -- executing a query just gave you an object that you iterate > over to get the results. It just seemed the most Pythonic way to > do it. Executing a query gives you some sort of object. That object either contains all the results, or has the resource handle (or whatever) needed to fetch more from the back end. That basically makes it a cursor, so we're not far different after all :) ChrisA From python at deborahswanson.net Thu Jun 1 10:58:27 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 1 Jun 2017 07:58:27 -0700 Subject: How to install Python package from source on Windows In-Reply-To: Message-ID: <005001d2dae7$877d6bd0$2860e747@sambora> Gregory Ewing wrote, on Thursday, June 01, 2017 12:59 AM > > Deborah Swanson wrote: > > I got one suggestion that I could just copy the files to "the > > appropriate directories", > > That was me, but I've just had a look at the source, and it > seems the core functionality is implemented in C, so scrub that idea. Yes, memoryslots.c is core functionality, and I was just going to have to fake it if the compiled file needed more than just being dropped into Site Packages. > But all is not lost. I've found this: > https://pypi.python.org/pypi/namedlist "Similar to namedtuple, but instances are mutable." It's all pure Python -- I checked this time! -- so you shouldn't have any trouble installing it. In fact it's just a single source file, namedlist.py. Put that in your site-packages folder and you should be good to go. -- Greg Thanks Greg! I just ran into another wall, unsolvable by any methods I've come up with yet, so I'm trying to unroll the namedtuples into a list of lists, and that turns out to be trickier than you'd think too. I'll give namedlist a try, seems the PyPi people have seen that hole in namedtuples' functionality and have worked on some alternatives. Spectacular! Deborah From python at deborahswanson.net Thu Jun 1 11:09:11 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 1 Jun 2017 08:09:11 -0700 Subject: How to install Python package from source on Windows In-Reply-To: Message-ID: <005701d2dae9$071ef470$2860e747@sambora> Gregory Ewing wrote, on Thursday, June 01, 2017 1:12 AM > > Deborah Swanson wrote: > > Why do you care so deeply what pip does on an operating > system that is > > no longer supported? > > Being pipless is a sufficiently distressing fate that we'd > like to help, even if you didn't explicitly request it. > > Particularly since it seems like such an unnecessary one, > because installing pip shouldn't require any kind of compiler > on any version of any operating system, supported or > otherwise. So the fact that it apparently does on your system > is very puzzling, and some of us would like to solve that puzzle. > > -- > Greg That's nice, but as I've said, the problem can be solved by reinstalling Anaconda3 and PyCharm. Not fun or easy, but at least the outcome would be known if I really needed pip, which I won't for quite awhile. Honestly, this installation of Anaconda3 is so unstable it behaves differently everytime I try to do something with it. Better to leave well enough alone right now. I really need what little usable time I have making progress with Python, and I should be a lot healthier in a few months. And I'd rather spend the downtime then getting my good computer up and running, and Python on Linux again. Then this problem vanishes. Deborah From python at deborahswanson.net Thu Jun 1 11:19:16 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 1 Jun 2017 08:19:16 -0700 Subject: How to install Python package from source on Windows In-Reply-To: Message-ID: <005d01d2daea$6f89a5e0$2860e747@sambora> Gregory Ewing wrote, on Thursday, June 01, 2017 1:15 AM > > Deborah Swanson wrote: > > I have already offered to do whatever you would like me to > do on this > > system > > If you wouldn't mind, I'd like to see the output from this command: > > python -m ensurepip > > -- > Greg The rest of that sentence was something like later, when I have Python in Linux and don't need this sytem anymore. I've tried ensurepip 2 or 3 times, with different unsuccessful outcomes each time. I really am done trying to fix this installation, and I just hope it holds together for PyCharm's use as long as I need it to. Couple months, I think. And if I don't touch it, should be ok. Now please, can we be done with this for now? Deborah From mirkok.lists at googlemail.com Thu Jun 1 11:26:19 2017 From: mirkok.lists at googlemail.com (Mirko) Date: Thu, 01 Jun 2017 17:26:19 +0200 Subject: [OT] How to improve my programming skills? Message-ID: <5930321B.9040405@googlemail.com> Hello everybody! TLDR: Sorry for OT. Long-time Linux geek and hobby programmer wants to improve his coding skills. What's most important: project planing, algorithms and data structures, contributing to FOSS, web development, learning other languages or something else? Sorry for posting such a (long) off-topic question, which even is partly even more about psychology, than about technology. But I'm mostly using Python and Bash for programming and am reading this mailing list/newsgroup for many years now, and hope it's Ok (feel free to direct me somewhere more appropriate, as long it's an equally helpful and friendly place). I'm looking for a way (*the* way, ie. the "BEST(tm)" way) to improve my coding skills. While I'm a quite hard-core computer geek since 25 years and a really good (hobbyist) Linux-SOHO-Admin, my programming skills are less than sub-par. I wish to change that and become at least am average hobby-programmer. I'm an excellent autodidact and in fact, all what I know about computers is completely self-taught. Now, the problem is, that I'm 41 years old now, have a day job (which hasn't much to do with advanced computing stuff), friends and all that. There is little time left for this undertaking (something like 5 - 10 hours per week), so I'm looking for the most efficient ways to do this. I identify the following problems which could be worth improving: - I never sit down and plan anything with pencil and paper, flowcharts, UML or anything else. I just fire up Vim and start typing. That works (albeit slowly) for simple programs, but as soon as the project becomes a little more complex the parts and components don't match well and I need to botch something together to somehow make it work. And in the resulting mess I lose the interest. - I never learned algorithms and data structures. I know *what* (linked) lists, dicts, arrays, structs, and trees are; what binary search or bubble-sort is, but I never really studied them, let alone implemented them for educative purposes. - When it comes to coding, I'm heavily shy and unsure. I really know my stuff when it's about Linux-SOHO-Administration, but when trying to contribute to FOSS projects I always hesitate for several reasons (somehow I never find those low-hanging fruits that everybody talks about; either it's super-easy or to difficult.) - For my day job (which is absolutely not my dream job) it would be quite useful to know web design/development, especially WordPress stuff. But web programming always feel like being trapped in a mangrove jungle, with all those browser-specialties, different and incompatible JS-engines, PHP versions and all that. I'm almost never in the mood to learn web development. - Beside Python and Bash -- which I both know rather well (for a hobbyist at least) -- I only know a little C, some LUA and a tiny bit of Scheme. - I'm very hard to motivate, when the issue or topic doesn't interest me much. I know some tricks to increase my motivation in such cases, but don't use them enough. What do you think, which of the problems would be most important to overcome? What would be the most efficient way for improving my general programming skills? Do you have any other suggestions or tips? Much thanks for your time! Mirko From ian.g.kelly at gmail.com Thu Jun 1 12:13:13 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 1 Jun 2017 10:13:13 -0600 Subject: Circular iteration on tuple starting from a specific index In-Reply-To: References: Message-ID: On Thu, Jun 1, 2017 at 4:14 AM, Gregory Ewing wrote: > guillaume.paulet at giome.fr wrote: >> >> def cycle_once(iterable, start): >> return chain(islice(iterable, start, None), islice(iterable, start)) This assumes that iterable is restartable, which is not the case if iterable is itself an iterator. > Another variation, maybe slightly more efficient: > > from itertools import islice, cycle > > def cycle_once(iterable, start): > return islice(cycle(iterable), start, start + len(iterable)) Similar problem here, len will fail on an iterator. I know the OP specified a tuple (and that was what my own solution assumed), but "iterable" implies broader applicability. From zljubisic at gmail.com Thu Jun 1 13:01:19 2017 From: zljubisic at gmail.com (zljubisic at gmail.com) Date: Thu, 1 Jun 2017 10:01:19 -0700 (PDT) Subject: Converting epoch to string in format yyyy-mm-dd, or maybe it is not necessary Message-ID: <92bbcf4a-07ec-4184-a8ed-5fbeb5e58d92@googlegroups.com> I have a dataframe with epoh dates, something like this: df = pd.DataFrame( { 'epoch' : [1493928008, 1493928067, 1493928127, 1493928310, 1493928428, 1493928547]}) I want to create a new column with epoch converted to yyyy-mm-dd as string. Actually, I have a epoch column, and I would like to use it for grouping data by day, let's say min, max... mayge there is no need for conversion at all. How to do that? From richardmoseley4 at gmail.com Thu Jun 1 13:44:10 2017 From: richardmoseley4 at gmail.com (Richard Moseley) Date: Thu, 1 Jun 2017 18:44:10 +0100 Subject: Test to see if message is bounced Message-ID: Apologies if this appears on the list, but I'm checking whether I have been placed onto a blacklist since previous postings have been bounced back with an error that the email address is not on the "subscribers list". I previously had an email address of "dickie.moseley at virgin.net" which has now been deactivated. Richard Moseley From rosuav at gmail.com Thu Jun 1 13:47:30 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Jun 2017 03:47:30 +1000 Subject: Test to see if message is bounced In-Reply-To: References: Message-ID: On Fri, Jun 2, 2017 at 3:44 AM, Richard Moseley wrote: > Apologies if this appears on the list, but I'm checking whether I have been > placed onto a blacklist since previous postings have been bounced back with > an error that the email address is not on the "subscribers list". I > previously had an email address of "dickie.moseley at virgin.net" which has > now been deactivated. Looks fine to me. Most likely, your *email address*, not you as a person, would have been blocked for excessive bounces. It's nothing against you as a person. ChrisA From python at mrabarnett.plus.com Thu Jun 1 16:22:59 2017 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 1 Jun 2017 21:22:59 +0100 Subject: [OT] How to improve my programming skills? In-Reply-To: <5930321B.9040405@googlemail.com> References: <5930321B.9040405@googlemail.com> Message-ID: On 2017-06-01 16:26, Mirko via Python-list wrote: [snip] > > I'm looking for a way (*the* way, ie. the "BEST(tm)" way) to improve > my coding skills. While I'm a quite hard-core computer geek since 25 > years and a really good (hobbyist) Linux-SOHO-Admin, my programming > skills are less than sub-par. I wish to change that and become at > least am average hobby-programmer. I'm an excellent autodidact and > in fact, all what I know about computers is completely self-taught. > > Now, the problem is, that I'm 41 years old now, have a day job > (which hasn't much to do with advanced computing stuff), friends and > all that. There is little time left for this undertaking (something > like 5 - 10 hours per week), so I'm looking for the most efficient > ways to do this. > The best way is practice, which takes time. I don't know of a shortcut. > I identify the following problems which could be worth improving: > > - I never sit down and plan anything with pencil and paper, > flowcharts, UML or anything else. I just fire up Vim and start > typing. That works (albeit slowly) for simple programs, but as soon > as the project becomes a little more complex the parts and > components don't match well and I need to botch something together > to somehow make it work. And in the resulting mess I lose the interest. > Flowcharts? UML? Never used them! :-) I _have_ used sometimes pseudocode, though. Python has been described as "executable pseudocode" in the past, which is probably why I like it so much. > - I never learned algorithms and data structures. I know *what* > (linked) lists, dicts, arrays, structs, and trees are; what binary > search or bubble-sort is, but I never really studied them, let alone > implemented them for educative purposes. > Higher-level languages like Python have dicts and sorting built-in, so unless you're working on improving the low-level implementation, you don't need to know the details. [snip] > > - I'm very hard to motivate, when the issue or topic doesn't > interest me much. I know some tricks to increase my motivation in > such cases, but don't use them enough. > You could pick something that you could automate. That way, you'd be learning and also saving time in the long run. > > > What do you think, which of the problems would be most important to > overcome? What would be the most efficient way for improving my > general programming skills? Do you have any other suggestions or tips? > If you don't enjoy programming, it's probably not for you! :-) From neilc at norwich.edu Thu Jun 1 16:51:01 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Thu, 1 Jun 2017 20:51:01 +0000 (UTC) Subject: [OT] How to improve my programming skills? References: <5930321B.9040405@googlemail.com> Message-ID: On 2017-06-01, Mirko via Python-list wrote: > Hello everybody! > > TLDR: Sorry for OT. Long-time Linux geek and hobby programmer > wants to improve his coding skills. What's most important: > project planing, algorithms and data structures, contributing > to FOSS, web development, learning other languages or something > else? I recommend a few things you haven't yet mentioned: learning to use source control, and working toward a systematic way of composing and runnings tests as you develop.. Secondly, I recommend picking up a programming langauge that concentrates on functional programming with immutable state if you haven't done it before. The book that worked for me was Simply Scheme https://people.eecs.berkeley.edu/~bh/ss-toc2.html, but that's sorta ancient history now and I'm sure there's lots more options out there. -- Neil Cerutti From rossiheron at gmail.com Thu Jun 1 19:44:03 2017 From: rossiheron at gmail.com (Heron Rossi) Date: Thu, 1 Jun 2017 16:44:03 -0700 (PDT) Subject: Celery + Django logging and RotateFileHandler Message-ID: I have a Django project setup with Celery and a RotatingFileHandler setting like shown below: LOGGING = { 'version': 1, 'handlers': { 'console':{ 'class':'logging.StreamHandler', 'stream': sys.stdout, }, 'celery': { 'class': 'logging.handlers.RotatingFileHandler', 'filename': '/var/log/celery/celery.log', 'formatter': 'verbose', 'maxBytes': (1024 * 1024 * 10), # 10 mb 'backupCount': 10 }, }, 'formatters': { 'verbose': { 'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s' }, }, 'loggers': { 'sm9.views': { 'handlers':['console'], 'propagate': True, 'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'), }, 'sm9.helpers': { 'handlers':['console'], 'propagate': True, 'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'), }, 'sm9.soap': { 'handlers':['celery'], 'propagate': True, 'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'), }, 'sm9.tasks': { 'handlers':['celery'], 'propagate': True, 'level': os.getenv('CELERYD_LOG_LEVEL', 'INFO'), }, 'celery': { 'handlers': ['celery'], 'level': os.getenv('CELERYD_LOG_LEVEL', 'INFO'), 'propagate': True }, }, } When celery.log reaches 10mb the system is creating 2 more files: celery.log.1, celery.log.2 . All files are beeing populated simultaneously instead of reaching 10mb and opening a new file. Is this the expected behavior? Celery is running as a daemon with 3 threads. From steve+python at pearwood.info Fri Jun 2 01:30:43 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 02 Jun 2017 15:30:43 +1000 Subject: [OT] How to improve my programming skills? References: <5930321B.9040405@googlemail.com> Message-ID: <5930f804$0$1588$c3e8da3$5496439d@news.astraweb.com> On Fri, 2 Jun 2017 01:26 am, Mirko wrote: > TLDR: Sorry for OT. Long-time Linux geek and hobby programmer wants > to improve his coding skills. What's most important: project > planing, algorithms and data structures, contributing to FOSS, web > development, learning other languages or something else? Programming. If you want to improve your programming skills, you must *write code* and then *run that code* and if it breaks you must *fix that code*. I cannot tell you how many hours I have wasted writing code that I've never actually run. Its a trap. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From dieter at handshake.de Fri Jun 2 02:20:06 2017 From: dieter at handshake.de (dieter) Date: Fri, 02 Jun 2017 08:20:06 +0200 Subject: Converting epoch to string in format yyyy-mm-dd, or maybe it is not necessary References: <92bbcf4a-07ec-4184-a8ed-5fbeb5e58d92@googlegroups.com> Message-ID: <87tw3ylxm1.fsf@handshake.de> zljubisic at gmail.com writes: > I have a dataframe with epoh dates, something like this: > > df = pd.DataFrame( { 'epoch' : [1493928008, 1493928067, 1493928127, 1493928310, 1493928428, 1493928547]}) > > I want to create a new column with epoch converted to yyyy-mm-dd as string. "epoch" dates usually mean seconds since 1.1.1970, represented as an integer or float. A single epoch date is usually not represented as a list (as above). For details, read the head of the "time" module documentation. The "time" module contains 2 functions to convert from an "epoch date" to a time tuple: "gmtime" (this gives an UTC or Greenwich Mean time tuple) and "localtime" (this gives a time tuple represented in the local time zone). You can use "time.strftime" to get a string representation (according to a format you specify) from a time tuple. Again, the documentation of the "time" module tells the details. From frank at chagford.com Fri Jun 2 03:18:10 2017 From: frank at chagford.com (Frank Millman) Date: Fri, 2 Jun 2017 09:18:10 +0200 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: "Skip Montanaro" wrote in message news:CANc-5Uz2ruXRWnAX8pJEVQZtQbndC0aOJz3gGeb04k1ZfFfQgQ at mail.gmail.com... > Assuming the underlying database supports transactions, is there any difference between calling the commit() method on the connection and calling the execute method on the cursor with the "commit transaction" statement? It seems a bit asymmetric to me to start a transaction with > cur.execute("begin transaction") > but end it with > conn.commit() Yes there is a difference, at least as far as the combination of PostgreSQL and psycopg2 is concerned. I will use 'PSQL' in the following, to save me some typing. A while ago I had to delve into PSQL locking, as I had a problem with locks not being cleared. I learned that, for a simple SELECT statement, PSQL checks to see if it is in a transaction. If not, it does not set any locks, but if it is, it creates a lock which is cleared on the next COMMIT/ROLLBACK. By default, psycopg2 uses 'autocommit', which means that even a SELECT is preceded by a 'BEGIN' statement internally. I never changed the default, so all of the following assumes that autocommit is on. I had many SELECT's, but I was not issuing any form of commit, so the locks built up. I solved my problem by always committing. However in my experimenting I found something curious. I had one window open on a python session, where I could execute commands, and another on a psql session, where I could monitor the 'lock' table. I found that, if I issued a SELECT, a lock was created, if I called conn.commit(), the lock was cleared. I could repeat this sequence and the pattern was consistent. However, if I issued a SELECT and called cur.execute('commit'), the lock was cleared, but the next SELECT did *not* create a lock. I worked out a possible reason for this, which I have not proved it by examining the source code of psycopg2, but is internally consistent. The theory goes like this - psycopg2 is in one of two states - a transaction is active, or it is not active. If you execute any command, and a transaction is not active, it starts a transaction first. If you call conn.commit() or conn.rollback(), it sends the command to the database and resets its state. However, (and this is the theory,) if you call cur.execute('commit'), it sends the command to the database, but does not reset its state. So when you execute the next command, it thinks the transaction is still active, so it does not start a new transaction. PSQL, on the other hand, knows that the previous transaction has been committed, so if the next command is a SELECT, it does not create a lock. As I said, I cannot prove this, but the theory fits the observed behaviour perfectly, so I have proceeded on the assumption that it is true. Therefore I now always run every SQL command or block of commands within a context manager, which always calls conn.commit() or conn.rollback() on exit, and I have not had any more problems. I use exactly the same code for sqlite3 and for Sql Server/pyodbc, and it has not caused any problems there either. Frank Millman From frank at chagford.com Fri Jun 2 03:24:58 2017 From: frank at chagford.com (Frank Millman) Date: Fri, 2 Jun 2017 09:24:58 +0200 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: "Frank Millman" wrote in message news:ogr3ff$sg1$1 at blaine.gmane.org... > By default, psycopg2 uses 'autocommit', which means that even a SELECT is preceded by a 'BEGIN' statement internally. I never changed the default, so all of the following assumes that autocommit is on. Oops - by default it does *not* use autocommit, so the following assumes that it is off. Frank From greg.ewing at canterbury.ac.nz Fri Jun 2 04:06:59 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 02 Jun 2017 20:06:59 +1200 Subject: Working with dictionaries and keys help please! In-Reply-To: <592fda25$0$1586$c3e8da3$5496439d@news.astraweb.com> References: <53455047-7b3a-4874-ab3d-ed02957b5c97@googlegroups.com> <592fda25$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: > On Thu, 1 Jun 2017 10:29 am, David D wrote: > >> Is there a way of performing this >> where the key will update so that is continues to work sequentially? It sounds like you don't want a dictionary at all, you want a list. You can use the index() method to find the current "key" of an entry. >>> people = ["John", "David", "Phil", "Bob"] >>> people.index("Phil") 2 >>> people.remove("David") >>> people.index("Phil") 1 -- Greg From rosuav at gmail.com Fri Jun 2 04:08:48 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Jun 2017 18:08:48 +1000 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: On Fri, Jun 2, 2017 at 5:18 PM, Frank Millman wrote: > As I said, I cannot prove this, but the theory fits the observed behaviour > perfectly, so I have proceeded on the assumption that it is true. Therefore > I now always run every SQL command or block of commands within a context > manager, which always calls conn.commit() or conn.rollback() on exit, and I > have not had any more problems. I use exactly the same code for sqlite3 and > for Sql Server/pyodbc, and it has not caused any problems there either. +1. A bit more info: When you perform read-only queries against a PostgreSQL database, you still have transactional integrity, just as you would with mutating transactions. Two SELECT statements in the same transaction will see a consistent view of the underlying database. To accomplish this, the database creates low-grade locks, so it knows which things you're using. (It's not quite that simple, since Postgres uses MVCC, but broadly speaking it's so.) Thus transactions are just as important for SELECT statements as they are for INSERT or UPDATE... or, for that matter, ALTER TABLE (this is a point on which not all DBMSes agree - transactional DDL is one of the features I love about Postgres). Always using a context manager is good practice and great for code clarity. I would be inclined to mandate it in a style guide, if I were in charge of any good-sized psycopg2-based project. ChrisA From greg.ewing at canterbury.ac.nz Fri Jun 2 04:10:51 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 02 Jun 2017 20:10:51 +1200 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: <3dvqic92ts7tei230l03hnhr1b51794fs2@4ax.com> <4cdric1iau91j5p32mecn407ta8a0ad8g4@4ax.com> Message-ID: Chris Angelico wrote: > Executing a query gives you some sort of object. That object either > contains all the results, or has the resource handle (or whatever) > needed to fetch more from the back end. That basically makes it a > cursor, so we're not far different after all :) The point is that my API doesn't make a big deal out of them. You don't typically think about them, just as you don't usually think much about the intermediate iterator created when you do "for x in some_list". -- Greg From greg.ewing at canterbury.ac.nz Fri Jun 2 04:15:42 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 02 Jun 2017 20:15:42 +1200 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: Frank Millman wrote: > I never changed the > default, so all of the following assumes that autocommit is on. > > I had many SELECT's, but I was not issuing any form of commit, so the > locks built up. I solved my problem by always committing. Something is screwy when a feature called "autocommit" results in you having to issue explicit commits. -- Greg From renting at astron.nl Fri Jun 2 04:16:03 2017 From: renting at astron.nl (Adriaan Renting) Date: Fri, 02 Jun 2017 10:16:03 +0200 Subject: [OT] How to improve my programming skills? In-Reply-To: <5930321B.9040405@googlemail.com> References: <5930321B.9040405@googlemail.com> Message-ID: <59313AE30200001B00017FC4@smtp1.astron.nl> Adriaan Renting | Email: renting at astron.nl Software Engineer Radio Observatory ASTRON | Phone: +31 521 595 100 (797 direct) P.O. Box 2 | GSM: +31 6 24 25 17 28 NL-7990 AA Dwingeloo | FAX: +31 521 595 101 The Netherlands | Web: http://www.astron.nl/~renting/ >>> On 1-6-2017 at 17:26, Mirko via Python-list wrote: > Hello everybody! > > TLDR: Sorry for OT. Long-time Linux geek and hobby programmer wants > to improve his coding skills. What's most important: project > planing, algorithms and data structures, contributing to FOSS, web > development, learning other languages or something else? > Learning languages is something that sort of happens by itself as times change. > > Sorry for posting such a (long) off-topic question, which even is > partly even more about psychology, than about technology. But I'm > mostly using Python and Bash for programming and am reading this > mailing list/newsgroup for many years now, and hope it's Ok (feel > free to direct me somewhere more appropriate, as long it's an > equally helpful and friendly place). > > I'm looking for a way (*the* way, ie. the "BEST(tm)" way) to improve > my coding skills. While I'm a quite hard-core computer geek since 25 > years and a really good (hobbyist) Linux-SOHO-Admin, my programming > skills are less than sub-par. I wish to change that and become at > least am average hobby-programmer. I'm an excellent autodidact and > in fact, all what I know about computers is completely self-taught. > The problem with being an autodidact is that you usually have big gaps in your knowledge that you're not aware of. The first question is if you want to be a better programmer or better software engineer? What I mean with that is do you want to make your code more robust and efficient or do you want to learn how to design better code? The first is very much about algorithms and the math behind them, while the second is more about tools and methods to help you design. > Now, the problem is, that I'm 41 years old now, have a day job > (which hasn't much to do with advanced computing stuff), friends and > all that. There is little time left for this undertaking (something > like 5 - 10 hours per week), so I'm looking for the most efficient > ways to do this. > If you have little time, it will take a longer time to get better. > I identify the following problems which could be worth improving: > > - I never sit down and plan anything with pencil and paper, > flowcharts, UML or anything else. I just fire up Vim and start > typing. That works (albeit slowly) for simple programs, but as soon > as the project becomes a little more complex the parts and > components don't match well and I need to botch something together > to somehow make it work. And in the resulting mess I lose the interest. > Any hour designing usually saves you many hours in coding. UML is popular, but I'm a fan of Yourdon as well. Design really starts with writing down what you're trying to do (use cases), why, how and within what limits. You get better through experience, especially when doing it for the first time, people often forget a lot of the limits they have. Part of it is also software management, in this case mostly on yourself. One of the books that's the foundation there is "The Mythical Man Month". It's old but it would help the industry if everyone had read it. > - I never learned algorithms and data structures. I know *what* > (linked) lists, dicts, arrays, structs, and trees are; what binary > search or bubble-sort is, but I never really studied them, let alone > implemented them for educative purposes. > You're touching on the difference between information and knowledge. You know they exist, but they're not really part of your knowledge. If you want to get better at programming, you need to learn more about algorithms, making excercises and actually implementing these will make them part of your knowledge. Some general math will also help, it really helps if you can analyse if the code you're writing uses a linear or quadratic solution to your problem, and how it could be rewritten. At a higher level, there are also things like design patterns. A standard book on that would be Design Patterns by Erich Gamma, Richard Helm, Ralph Johnson en John Vlissides. > - When it comes to coding, I'm heavily shy and unsure. I really know > my stuff when it's about Linux-SOHO-Administration, but when trying > to contribute to FOSS projects I always hesitate for several reasons > (somehow I never find those low-hanging fruits that everybody talks > about; either it's super-easy or to difficult.) > Sounds like you have typical sysadmin skills, but not really programming or software engineering skills. > - For my day job (which is absolutely not my dream job) it would be > quite useful to know web design/development, especially WordPress > stuff. But web programming always feel like being trapped in a > mangrove jungle, with all those browser-specialties, different and > incompatible JS-engines, PHP versions and all that. I'm almost never > in the mood to learn web development. > Any programming outside academia needs to interface with the real world and thus will encounter stuff like that. > - Beside Python and Bash -- which I both know rather well (for a > hobbyist at least) -- I only know a little C, some LUA and a tiny > bit of Scheme. > I found it quite insightful to learn a language where you have to do your own memory management, and a low level assembly language as it really helps in understanding how to write efficient software, even if you don't use it much afterwards. > - I'm very hard to motivate, when the issue or topic doesn't > interest me much. I know some tricks to increase my motivation in > such cases, but don't use them enough. > > You need to find a problem you really want to solve. I started out writing tools for games as a teenager because I wanted to have Simcity, Civilization or VGA Planets do things. > > What do you think, which of the problems would be most important to > overcome? What would be the most efficient way for improving my > general programming skills? Do you have any other suggestions or tips? > The most efficient way to learn is by being tought by a good teacher. > > Much thanks for your time! > > Mirko From rosuav at gmail.com Fri Jun 2 04:24:28 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Jun 2017 18:24:28 +1000 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: <3dvqic92ts7tei230l03hnhr1b51794fs2@4ax.com> <4cdric1iau91j5p32mecn407ta8a0ad8g4@4ax.com> Message-ID: On Fri, Jun 2, 2017 at 6:10 PM, Gregory Ewing wrote: > Chris Angelico wrote: >> >> Executing a query gives you some sort of object. That object either >> contains all the results, or has the resource handle (or whatever) >> needed to fetch more from the back end. That basically makes it a >> cursor, so we're not far different after all :) > > > The point is that my API doesn't make a big deal out of them. > You don't typically think about them, just as you don't usually > think much about the intermediate iterator created when you > do "for x in some_list". Which is a fully reasonable way to do things. And actually, on further consideration, I think it's probably the better way; the number of times you would want to explicitly create a cursor are so few that they can be handled by some sort of unusual method call, and the basic usage should look something like this: with psycopg2.connect(...) as conn: with conn.trans() as trn: for row in trn.execute("select ..."): print(row) The outer context manager is optional, but not the inner one and the method call, as I'm not a fan of the unusual usage where "with conn:" creates a transaction - it's different from the way most context managers work (managing the resource represented by the object itself, not some additional resource allocated on __enter__). The iterator used on trn.execute would be a cursor such as you describe. ChrisA From guillaume.paulet at giome.fr Fri Jun 2 04:30:02 2017 From: guillaume.paulet at giome.fr (guillaume.paulet at giome.fr) Date: Fri, 02 Jun 2017 10:30:02 +0200 Subject: Circular iteration on tuple starting from a specific index Message-ID: @Gregory Ewing: you were right, your version without *chain* is faster and I quiet like it :) ``` from timeit import timeit from itertools import chain, cycle, islice def cycle_once_with_chain(sequence, start): return chain(islice(sequence, start, None), islice(sequence, start)) def cycle_once_without_chain(sequence, start): return islice(cycle(sequence), start, start + len(sequence)) sequence = tuple(i for i in range(100)) time_with_chain = timeit( stmt='cycle_once_with_chain(sequence, 50)', number=1000000, globals=globals() ) print('Method with *chain* took: ', (time_with_chain /1000000), ' per call.') # Method with *chain* took: 5.025957580000977e-07 per call. time_without_chain = timeit( stmt='cycle_once_without_chain(sequence, 50)', number=1000000, globals=globals() ) print('Method without *chain* took: ', (time_without_chain /1000000), ' per call.') #Method without *chain* took: 3.5880194699984714e-07 per call. ``` @Ian: Good point here, these two methods only works with sequences (list, tuple, string...). I renamed it appropriately in the above sample code :) From jon+usenet at unequivocal.eu Fri Jun 2 06:46:46 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Fri, 2 Jun 2017 10:46:46 -0000 (UTC) Subject: Python DB API - commit() v. execute("commit transaction")? References: Message-ID: On 2017-06-02, Frank Millman wrote: > "Frank Millman" wrote in message news:ogr3ff$sg1$1 at blaine.gmane.org... > >> By default, psycopg2 uses 'autocommit', which means that even a SELECT is >> preceded by a 'BEGIN' statement internally. I never changed the default, so >> all of the following assumes that autocommit is on. > > Oops - by default it does *not* use autocommit, so the following assumes > that it is off. Indeed, the DB-API spec says that auto-commmit must be initially off. This led to an extremely irritating situation whereby Python-MySQLdb changed incompatibly between versions, it used to have auto-commit on but was changed to bring it in line with the spec - and they didn't even add any way of achieving the old backwards-compatible behaviour! (You can call Connection.autocommit() but this has to happen after the connection has already been established, and results in every new connection starting with two completely pointless "SET autocommit 0" "SET autocommit 1" commands.) From neilc at norwich.edu Fri Jun 2 08:07:45 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Fri, 2 Jun 2017 12:07:45 +0000 (UTC) Subject: Python DB API - commit() v. execute("commit transaction")? References: Message-ID: On 2017-06-02, Frank Millman wrote: > As I said, I cannot prove this, but the theory fits the > observed behaviour perfectly, so I have proceeded on the > assumption that it is true. Therefore I now always run every > SQL command or block of commands within a context manager, > which always calls conn.commit() or conn.rollback() on exit, > and I have not had any more problems. I use exactly the same > code for sqlite3 and for Sql Server/pyodbc, and it has not > caused any problems there either. You're probably not expected to interleave transaction control commands from different levels of abstraction, e.g., only call 'commit' directly if you called 'begin' directly. -- Neil Cerutti From jon+usenet at unequivocal.eu Fri Jun 2 12:45:40 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Fri, 2 Jun 2017 16:45:40 -0000 (UTC) Subject: Python DB API - commit() v. execute("commit transaction")? References: Message-ID: On 2017-06-02, Dennis Lee Bieber wrote: > Connector/Python (MySQL) [guess it is time for me to finally upgrade to > Python 3.x -- it was the delay in getting mysqldb ported that held me back] > does allow for turning on autocommit -- which is documented as issuing an > implicit commit after each SQL (which I take to mean each .execute() ), and > would likely cause problems with explicit BEGIN. Also not recommended for > InnoDB tables, but considered appropriate for MyISAM tables [no transaction > feature on those]. Bewaare - MyISAM tables have no transactions for DML but they do have transactions for DDL. Insane but true. From lele at metapensiero.it Fri Jun 2 12:55:15 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Fri, 02 Jun 2017 18:55:15 +0200 Subject: Manager for project templates, that allows "incremental" feature addition References: <10cc3f5f-9345-4db7-846b-cd912a3771a9@googlegroups.com> <871stpzm23.fsf@metapensiero.it> <261b8b58-e013-4d76-9e30-3335dfcd0d54@googlegroups.com> <87lgrugaft.fsf@metapensiero.it> Message-ID: <878tla2uto.fsf@metapensiero.it> Lele Gaifax writes: > Paul Moore writes: > >> On Thursday, 23 March 2017 15:56:43 UTC, Paul Moore wrote: >> >> Sadly, it doesn't support Windows, which is what I use. > > FYI, I just saw https://pypi.python.org/pypi/whaaaaat/0.4.0, that seems an > alternative port of Inquirer.js. Unfortunately it's Py2 only :-\ I contributed support for Python 3, so whaaaaat 0.5.2 is usable with modern snakes! > If you can try it under Windows, maybe I could be convinced in contributing > some Py3 compatibility fixes, and try to support that too in my tinject. I released https://pypi.python.org/pypi/metapensiero.tool.tinject/0.7, now using whaaaaat: I'd be curious to know if it works under Windows. ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From sean.dizazzo at gmail.com Fri Jun 2 13:17:05 2017 From: sean.dizazzo at gmail.com (sean.dizazzo at gmail.com) Date: Fri, 2 Jun 2017 10:17:05 -0700 (PDT) Subject: Bug or intended behavior? Message-ID: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Can someone please explain this to me? Thanks in advance! ~Sean Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> print "foo %s" % 1-2 Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for -: 'str' and 'int' >>> From jussi.piitulainen at helsinki.fi Fri Jun 2 13:28:28 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Fri, 02 Jun 2017 20:28:28 +0300 Subject: Bug or intended behavior? References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: sean.dizazzo at gmail.com writes: > Can someone please explain this to me? Thanks in advance! > > ~Sean > > > Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) > [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> print "foo %s" % 1-2 > Traceback (most recent call last): > File "", line 1, in > TypeError: unsupported operand type(s) for -: 'str' and 'int' >>>> The per cent operator has precedence over minus. Spacing is not relevant. Use parentheses. From irmen.NOSPAM at xs4all.nl Fri Jun 2 13:31:44 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Fri, 2 Jun 2017 19:31:44 +0200 Subject: Bug or intended behavior? In-Reply-To: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: <5931a100$0$791$e4fe514c@news.xs4all.nl> On 2-6-2017 19:17, sean.dizazzo at gmail.com wrote: > Can someone please explain this to me? Thanks in advance! > > ~Sean > > > Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) > [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> print "foo %s" % 1-2 > Traceback (most recent call last): > File "", line 1, in > TypeError: unsupported operand type(s) for -: 'str' and 'int' >>>> The % operator has higher precedence than the - operator. See https://docs.python.org/3/reference/expressions.html#operator-precedence So what you wrote is equivalent to: print ("foo %s" % 1) - 2 which means subtract the number 2 from the string "foo 1". Hence the error. Solution is to use parentheses to make sure the things are evaluated in the order you intended: print "foo %s" % (1-2) Irmen From walters.justin01 at gmail.com Fri Jun 2 13:34:10 2017 From: walters.justin01 at gmail.com (justin walters) Date: Fri, 2 Jun 2017 10:34:10 -0700 Subject: Bug or intended behavior? In-Reply-To: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On Fri, Jun 2, 2017 at 10:17 AM, wrote: > Can someone please explain this to me? Thanks in advance! > > ~Sean > > > Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) > [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> print "foo %s" % 1-2 > Traceback (most recent call last): > File "", line 1, in > TypeError: unsupported operand type(s) for -: 'str' and 'int' > >>> > -- > https://mail.python.org/mailman/listinfo/python-list > Try this: print "foo %d" % (1-2) From bgailer at gmail.com Fri Jun 2 13:45:41 2017 From: bgailer at gmail.com (bob gailer) Date: Fri, 2 Jun 2017 13:45:41 -0400 Subject: Bug or intended behavior? In-Reply-To: References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: <5d0eaeff-55b1-5989-df92-6f3c2f6a33e7@gmail.com> On 6/2/2017 1:28 PM, Jussi Piitulainen wrote: > sean.dizazzo at gmail.com writes: > >> Can someone please explain this to me? Thanks in advance! >> >> ~Sean >> >> >> Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) >> [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin >> Type "help", "copyright", "credits" or "license" for more information. >>>>> print "foo %s" % 1-2 >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: unsupported operand type(s) for -: 'str' and 'int' > The per cent operator has precedence over minus. Spacing is not > relevant. Use parentheses. In other words "foo %s" % 1 is executed, giving "1". Then "1"-2 is attempted giving the error. Also: If there is more than one conversion specifier the right argument to % must be a tuple. I usually write a tuple even if there is only one conversion specifier - that avoids the problem you encountered and makes it easy to add more values when you add more conversion specifiers. print "foo %s" % (1-2,) Bob Gailer From neilc at norwich.edu Fri Jun 2 13:50:29 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Fri, 2 Jun 2017 17:50:29 +0000 (UTC) Subject: Python DB API - commit() v. execute("commit transaction")? References: Message-ID: On 2017-06-02, Dennis Lee Bieber wrote: > > A bit of a long free-association rambling... > > On Fri, 2 Jun 2017 12:07:45 +0000 (UTC), Neil Cerutti > declaimed the following: >>You're probably not expected to interleave transaction control >>commands from different levels of abstraction, e.g., only call >>'commit' directly if you called 'begin' directly. > > .execute("begin") > is likely not safe either. > > If the adapter has been set to "autocommit", it might issue an > implicit "commit" after processing that execute -- wiping out > the transaction one has explicitly started... > > If not in "autocommit", the adapter may (will) at some point > issue an implicit "begin" -- resulting in an attempt to nest > transactions within the one connection. > > My conclusion: > If using a DB-API compliant adapter, explicitly issuing "begin" and > "commit" via .execute() should be avoided if one expects to be portable > (change the adapter from one DBMS to another). > Learn the behavior of the adapter (does any SQL start a transaction, or > only INSERT/UPDATE/DELETE/REPLACE -- the latter seems to be the > current SQLite3 documented behavior, exclusive of both editions > of the "Definitive Guide" which imply that an active > transaction will be commited upon executing a SELECT [Python > help file for module states that SELECT does /not/ commit]) so > you understand when it should be IN or OUT of a transaction > state. * Good point! > * Mixing various SQLite3 documentation (both the engine and Python's > module) gives a confusing mix: > The engine (per "Definite Guide") normally runs in autocommit -- and > appears to only go into non-autocommit when a "begin" is issued. > The module (per DB-API) runs in non-autocommit -- and issues an > implicit "begin" on the first of those DML operations mentioned above. > So... SELECT prior to any of the listed operations is effectively > auto-commit, as are any DDL operations (with the addition that DDL will > perform a commit IF the module believes a transaction is open). You configure the BEGIN operation by setting isolation_level. Setting it to IMMEDIATE (or EXCLUSIVE) avoids the deferral of lock acquisition. > Given the two -- turning on autocommit in the module may result > in no implicit "begin"; and transaction control is totally up > to the user .execute("begin|commit"). Agreed. > But this behavior may not match up with /other/ adapters, in > which turning ON autocommit in the adapter could just mean it > does a sequence of begin/SQL/commit for every .execute(). (per > documentation, not experience) sqlite3 behavior in autocommit matches up except when I explicitly muck things up with an explicit BEGIN. Conclusion seems to be that sqlite3 has a mode that permits explicit BEGIN/COMMIT, but you shouldn't do it *except* in that mode, and it's not portable. -- Neil Cerutti From rosuav at gmail.com Fri Jun 2 13:59:03 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 3 Jun 2017 03:59:03 +1000 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: On Sat, Jun 3, 2017 at 2:45 AM, Jon Ribbens wrote: > On 2017-06-02, Dennis Lee Bieber wrote: >> Connector/Python (MySQL) [guess it is time for me to finally upgrade to >> Python 3.x -- it was the delay in getting mysqldb ported that held me back] >> does allow for turning on autocommit -- which is documented as issuing an >> implicit commit after each SQL (which I take to mean each .execute() ), and >> would likely cause problems with explicit BEGIN. Also not recommended for >> InnoDB tables, but considered appropriate for MyISAM tables [no transaction >> feature on those]. > > Bewaare - MyISAM tables have no transactions for DML but they do have > transactions for DDL. Insane but true. Not insane; not all DBMSes have transactional DDL, and of the major ones, several have only more recently added it (despite having had rock-solid transactional DML for decades). It's an excellent feature but not easy to implement. Personally, I like it enough that I choose a DBMS based on features like that, but there are plenty of people who aren't too bothered by it. That said, though, MySQL is AFAIK the only full-scale DBMS that doesn't support DDL rollback in 2017. I don't know whether you can craft a transaction that mixes DDL and DML in all of them, but certainly in most; and it's a great feature, because you can version your schema trivially (putting a version number in a metadata table). I love it. :) ChrisA From remmmav at gmail.com Fri Jun 2 14:14:31 2017 From: remmmav at gmail.com (remmm) Date: Fri, 2 Jun 2017 11:14:31 -0700 (PDT) Subject: Very Slow Disk Writes when Writing Large Data Blocks Message-ID: <8c383179-4396-4efd-bed2-aa8e63433226@googlegroups.com> I'm seeing slow write speeds from both Python and C code on some Windows workstations. In particular both Python "write" and numpy "tofile" method suffers from this issue. I'm wondering if anyone has any ideas regarding if this is a known issue, know the cause, or how to resolve the issue? The details are below. The slow write speed issue seems to occur when writing data in blocks of larger than 32767 512-byte disk sectors. In terms of speed, write speed seems as expected until one gets to this 32767 limit and then the speed falls off as if all data beyond this is processed byte-by-byte. I can't prove this is what is happening -- but speed tests generally support this theory. These write speeds are in the range of 18 to 25 MBytes per second for spinning disks and about 50 Mbytes/sec for SSDs. Keep in mind these numbers should be more like 120 MBytes/sec for spinning disks and 300 MBytes/sec for SSDs. This issue seems to be system specific. I originally saw this on my HP z640 workstation using Python 2.7 under Windows 7. Originally it was numpy writes of large arrays in the 100GB size range that highlighted the issue, but I've since written test code using just python "write" too and get similar results using various block sizes. I've since verified this using cygwin mingw64 C and with Visual Studio C 2013. I've also tested this on a variety of other systems. My laptop does not show this speed issue, and not all z640 systems seem to show this issue though I've found several that do. IT has tested this on a clean Windows 7 image and on a Windows 10 image using yet another Z640 system and they get similar results. I've also not seen any Linux systems show this issue though I don't have any Z640's with Linux on them. I have however run my tests on Linux Mint 17 running under VirtualBox on the same Z640 that showed the speed issue and using both Wine and native python and both showed good performance and no slowdown. A work around for this seems to be to enable full caching for the drive in device manager with the subsequent risk of data corruption. This suggests for example that the issue is byte-by-byte flushing of data beyond the 32767 limit and that perhaps full cashing mitigates this some how. The other work around is to write all data in blocks of less than the 32767 limit (which is about 16Mbytes) as mentioned above. Of course reducing block size only works if you have the source code and the time and inclination to modify it. There is an indication that some of the commercial code we use for science and engineering also may suffer from this issue. The impact of this issue also seems application specific. The issue only becomes annoying when your regularity writing files of significant size (above say 10GB). It also depends on how an application writes data, so not all applications that create large files may exhibit this issue. As an example, Python numpy tofile method has this issue for large enough arrays and is the reason I started to investigate. I don't really know where to go with this. Is this a Windows issue? Is it an RTL issue? Is it a hardware, device driver, or bios issue? Is there a stated OS or library limit to buffer sizes to things like C fwrite or Python write which makes this an application issue? Thoughts? Thanks, remmm From skip.montanaro at gmail.com Fri Jun 2 14:55:49 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Fri, 2 Jun 2017 13:55:49 -0500 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: On Fri, Jun 2, 2017 at 11:14 AM, Dennis Lee Bieber wrote: > My conclusion: > If using a DB-API compliant adapter, explicitly issuing "begin" and > "commit" via .execute() should be avoided if one expects to be portable > (change the adapter from one DBMS to another). > Learn the behavior of the adapter (does any SQL start a > transaction, or > only INSERT/UPDATE/DELETE/REPLACE -- the latter seems to be the current > SQLite3 documented behavior, exclusive of both editions of the "Definitive > Guide" which imply that an active transaction will be commited upon > executing a SELECT [Python help file for module states that SELECT does > /not/ commit]) so you understand when it should be IN or OUT of a > transaction state. * > I just checked, and the sqlite3 adapter I have access to (Python 2.7.13 in a Conda env, module version 2.6.0, SQLite3 3.13.0) has no autocommit attribute at all. I checked at the module, connection and cursor levels. I'm using pyodbc via another layer added by others at work to connect to SQL Server. That extra layer explicitly sets autocommit to True on the underlying Connection object before returning it to the caller. In my case, my code isn't terribly large. I think it's safer to set autocommit to False and be explicit in my use of transactions. Skip From irmen.NOSPAM at xs4all.nl Fri Jun 2 15:29:10 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Fri, 2 Jun 2017 21:29:10 +0200 Subject: Very Slow Disk Writes when Writing Large Data Blocks In-Reply-To: <8c383179-4396-4efd-bed2-aa8e63433226@googlegroups.com> References: <8c383179-4396-4efd-bed2-aa8e63433226@googlegroups.com> Message-ID: <5931bc86$0$740$e4fe514c@news.xs4all.nl> On 2-6-2017 20:14, remmm wrote: > These write speeds are in the range of 18 to 25 MBytes per second for spinning disks and about 50 Mbytes/sec for SSDs. Keep in mind these numbers should be more like 120 MBytes/sec for spinning disks and 300 MBytes/sec for SSDs. You'll only reach those numbers in the ideal situation. Is there just one program doing this disk i/o, sequentially, from a single thread? If it is not, you are probably suffering disk i/o trashing once you filled up the drive's cache buffers. For example using Crystal Disk Mark on one of my HDD drives it reports max 60 MBytes/sec write speed sequentially in the ideal case (ok, it is not a very fast drive...) but only 0.7 (!) in the random 4k block case. Apparently Linux deals with this better than Windows, for your situation. Other than that the only other thing I can think of is interference of other programs on the system, such as malware protection or anti-virus tooling that is trying to scan your big files at the same time. That should be visible in Window's resource monitor tool however, I think. > I've since written test code using just python "write" Post it somewhere? Irmen From jon+usenet at unequivocal.eu Fri Jun 2 15:31:14 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Fri, 2 Jun 2017 19:31:14 -0000 (UTC) Subject: Python DB API - commit() v. execute("commit transaction")? References: Message-ID: On 2017-06-02, Chris Angelico wrote: > On Sat, Jun 3, 2017 at 2:45 AM, Jon Ribbens wrote: >> Bewaare - MyISAM tables have no transactions for DML but they do have >> transactions for DDL. Insane but true. > > Not insane; not all DBMSes have transactional DDL, and of the major > ones, several have only more recently added it (despite having had > rock-solid transactional DML for decades). It's an excellent feature > but not easy to implement. I'm not saying that transactional DDL is insane (it isn't), but MyISAM tables having transactions *only* for DDL is... surprising. Especially when it suddenly appeared as a "feature" in between two versions. It took me quite a while to work out why our database was randomly hanging. From neilc at norwich.edu Fri Jun 2 15:40:54 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Fri, 2 Jun 2017 19:40:54 +0000 (UTC) Subject: Python DB API - commit() v. execute("commit transaction")? References: Message-ID: On 2017-06-02, Skip Montanaro wrote: > On Fri, Jun 2, 2017 at 11:14 AM, Dennis Lee Bieber > wrote: > I just checked, and the sqlite3 adapter I have access to > (Python 2.7.13 in a Conda env, module version 2.6.0, SQLite3 > 3.13.0) has no autocommit attribute at all. I checked at the > module, connection and cursor levels. You get autocommit with sqlite3 by setting isolation_level=None on the connection object. https://docs.python.org/2/library/sqlite3.html#sqlite3-controlling-transactions -- Neil Cerutti From israel at ravnalaska.net Fri Jun 2 15:58:01 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Fri, 2 Jun 2017 11:58:01 -0800 Subject: CherryPy Session object creation logic Message-ID: I have a CherryPy app, for which I am using a PostgreSQL session. To be more exact, I modified a MySQL session class I found to work with PostgreSQL instead, and then I put this line in my code: cherrypy.lib.sessions.PostgresqlSession = PostgreSQLSession And this works fine. One thing about its behavior is bugging me, however: accessing a page instantiates (and deletes) *many* instances of this class, all for the same session. Doing some debugging, I counted 21 calls to the __init__ function when loading a single page. Logging in and displaying the next page hit it an additional 8 times. My theory is that essentially every time I try to read from or write to the session, CherryPy is instantiating a new PostgreSQLSession object, performing the request, and deleting the session object. In that simple test, that means 29 connections to the database, 29 instantiations, etc - quite a bit of overhead, not to mention the load on my database server making/breaking those connections (although it handles it fine). Is this "normal" behavior? Or did I mess something up with my session class? I'm thinking that ideally CherryPy would only create one object - and therefore, one DB connection - for a given session, and then simply hold on to that object until that session expired. But perhaps not? ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From skip.montanaro at gmail.com Fri Jun 2 16:24:39 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Fri, 2 Jun 2017 15:24:39 -0500 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: On Fri, Jun 2, 2017 at 2:40 PM, Neil Cerutti wrote: > You get autocommit with sqlite3 by setting isolation_level=None > on the connection object. > Thanks for the pointer. I'd probably never have noticed the correspondence. Skip From rosuav at gmail.com Fri Jun 2 16:48:28 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 3 Jun 2017 06:48:28 +1000 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: On Sat, Jun 3, 2017 at 5:31 AM, Jon Ribbens wrote: > On 2017-06-02, Chris Angelico wrote: >> On Sat, Jun 3, 2017 at 2:45 AM, Jon Ribbens wrote: >>> Bewaare - MyISAM tables have no transactions for DML but they do have >>> transactions for DDL. Insane but true. >> >> Not insane; not all DBMSes have transactional DDL, and of the major >> ones, several have only more recently added it (despite having had >> rock-solid transactional DML for decades). It's an excellent feature >> but not easy to implement. > > I'm not saying that transactional DDL is insane (it isn't), but MyISAM > tables having transactions *only* for DDL is... surprising. Especially > when it suddenly appeared as a "feature" in between two versions. It > took me quite a while to work out why our database was randomly hanging. Wait, you have transactions with MyISAM now? I thought MySQL supported transactions with InnoDB but not MyISAM, and the reason you didn't get transactional DDL was that the system catalog tables are mandatorily MyISAM, even if all your own tables are InnoDB. ChrisA From 6hktt.com at gmail.com Fri Jun 2 17:15:13 2017 From: 6hktt.com at gmail.com (6hktt.com at gmail.com) Date: Fri, 2 Jun 2017 14:15:13 -0700 (PDT) Subject: =?UTF-8?B?SEQgTElWRSDZhdiz2YTYs9mEINi62LHYp9io2YrYqCDYs9mI2K8g2KfZhNit2YTZgtipIA==?= =?UTF-8?B?OCBAKSApKU1CQygoINmF2LTYp9mH2K/YqSDYutix2KfYqNmK2Kgg2LPZiNivINin2YTYrdmE2YLYqSA4?= =?UTF-8?B?INin2YTYq9in2YXZhtipINmD2KfZhdmE2KkgTEJD?= Message-ID: <15f38913-8762-462b-8397-9aae47979c19@googlegroups.com> HD LIVE ????? ?????? ??? ?????? 8 @) ))MBC(( ?????? ?????? ??? ?????? 8 ??????? ????? LBCHD LIVE ????? ?????? ??? ?????? 8 @) ))MBC(( ?????? ?????? ??? ?????? 8 ??????? ????? LBCHD LIVE ????? ?????? ??? ?????? 8 @) ))MBC(( ?????? ?????? ??? ?????? 8 ??????? ????? LBCHD LIVE ????? ?????? ??? ?????? 8 @) ))MBC(( ?????? ?????? ??? ?????? 8 ??????? ????? LBCHD LIVE ????? ?????? ??? ?????? 8 @) ))MBC(( ?????? ?????? ??? ?????? 8 ??????? ????? LBCHD LIVE ????? ?????? ??? ?????? 8 @) ))MBC(( ?????? ?????? ??? ?????? 8 ??????? ????? LBC ?????? ??? ?????? 8 ?????? ??? ?????? 8 ????? ?????? ??? ?????? 8 ?????? ?????? ??? ?????? 8 ?????? ? ????? ?????? ??? ?????? 8 ???? ?? ??? https://goo.gl/WCZEeQ ?????? ??? ?????? 8 ?????? ??? ?????? 8 ????? ?????? ??? ?????? 8 ?????? ?????? ??? ?????? 8 ?????? ? ????? ?????? ??? ?????? 8 ???? ?? ??? https://goo.gl/WCZEeQ ?????? ??? ?????? 8 ?????? ??? ?????? 8 ????? ?????? ??? ?????? 8 ?????? ?????? ??? ?????? 8 ?????? ? ????? ?????? ??? ?????? 8 ???? ?? ??? https://goo.gl/WCZEeQ ?????? ??? ?????? 8 ?????? ??? ?????? 8 ????? ?????? ??? ?????? 8 ?????? ?????? ??? ?????? 8 ?????? ? ????? ?????? ??? ?????? 8 ???? ?? ??? https://goo.gl/WCZEeQ ?????? ??? ?????? 8 ?????? ??? ?????? 8 ????? ?????? ??? ?????? 8 ?????? ?????? ??? ?????? 8 ?????? ? ????? ?????? ??? ?????? 8 ???? ?? ??? https://goo.gl/WCZEeQ From rosuav at gmail.com Fri Jun 2 18:04:39 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 3 Jun 2017 08:04:39 +1000 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: On Sat, Jun 3, 2017 at 7:29 AM, Dennis Lee Bieber wrote: > On Sat, 3 Jun 2017 06:48:28 +1000, Chris Angelico > declaimed the following: > >> >>Wait, you have transactions with MyISAM now? I thought MySQL supported >>transactions with InnoDB but not MyISAM, and the reason you didn't get >>transactional DDL was that the system catalog tables are mandatorily >>MyISAM, even if all your own tables are InnoDB. >> > > Not really transactions -- but locks on the "metadata" tables... > > http://www.chriscalender.com/tag/myisam-locks/ Oh. That's just the basic protection of "don't let anyone change the table while we're using it". It doesn't mean you can roll back an ALTER TABLE, much less take advantage of full transactional integrity. In PostgreSQL, you can do something like this (pseudocode): version = #select schema_version from metadata# if version < 1: #create table foo (id serial primary key, bar text not null)# if version < 2: #alter table foo add quux integer not null default 10# if version < 3: #create table spam (id serial primary key, foo_id int not null references foo)# #update metadata set schema_version = 3# if version > 3: raise IntegrityError("Cannot backlevel database") #commit# Now, even if anything crashes out while you're migrating the database (either because the power fails, or because of an error in your code, or anything), you have an absolute guarantee that the version field and the database will be consistent - that version 2 *always* has both bar and quux columns, etc. There's no way to have half a schema migration done, or finish the migration but fail to update the version marker, or anything. You KNOW that it's safe, even against logic errors. That's what transactional DDL gives you. ChrisA From israel at ravnalaska.net Fri Jun 2 19:06:42 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Fri, 2 Jun 2017 15:06:42 -0800 Subject: Psycopg2 pool clarification Message-ID: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> I've been using the psycopg2 pool class for a while now, using code similar to the following: >>> pool=ThreadedConnectionPool(0,5,) >>> conn1=pool.getconn() >>> >>> pool.putconn(conn1) .... repeat later, or perhaps "simultaneously" in a different thread. and my understanding was that the pool logic was something like the following: - create a "pool" of connections, with an initial number of connections equal to the "minconn" argument - When getconn is called, see if there is an available connection. If so, return it. If not, open a new connection and return that (up to "maxconn" total connections) - When putconn is called, return the connection to the pool for re-use, but do *not* close it (unless the close argument is specified as True, documentation says default is False) - On the next request to getconn, this connection is now available and so no new connection will be made - perhaps (or perhaps not), after some time, unused connections would be closed and purged from the pool to prevent large numbers of only used once connections from laying around. However, in some testing I just did, this doesn't appear to be the case, at least based on the postgresql logs. Running the following code: >>> pool=ThreadedConnectionPool(0,5,) >>> conn1=pool.getconn() >>> conn2=pool.getconn() >>> pool.putconn(conn1) >>> pool.putconn(conn2) >>> conn3=pool.getconn() >>> pool.putconn(conn3) produced the following output in the postgresql log: 2017-06-02 14:30:26 AKDT LOG: connection received: host=::1 port=64786 2017-06-02 14:30:26 AKDT LOG: connection authorized: user=logger database=flightlogs 2017-06-02 14:30:35 AKDT LOG: connection received: host=::1 port=64788 2017-06-02 14:30:35 AKDT LOG: connection authorized: user=logger database=flightlogs 2017-06-02 14:30:46 AKDT LOG: disconnection: session time: 0:00:19.293 user=logger database=flightlogs host=::1 port=64786 2017-06-02 14:30:53 AKDT LOG: disconnection: session time: 0:00:17.822 user=logger database=flightlogs host=::1 port=64788 2017-06-02 14:31:15 AKDT LOG: connection received: host=::1 port=64790 2017-06-02 14:31:15 AKDT LOG: connection authorized: user=logger database=flightlogs 2017-06-02 14:31:20 AKDT LOG: disconnection: session time: 0:00:05.078 user=logger database=flightlogs host=::1 port=64790 Since I set the maxconn parameter to 5, and only used 3 connections, I wasn't expecting to see any disconnects - and yet as soon as I do putconn, I *do* see a disconnection. Additionally, I would have thought that when I pulled connection 3, there would have been two connections available, and so it wouldn't have needed to connect again, yet it did. Even if I explicitly say close=False in the putconn call, it still closes the connection and has to open What am I missing? From this testing, it looks like I get no benefit at all from having the connection pool, unless you consider an upper limit to the number of simultaneous connections a benefit? :-) Maybe a little code savings from not having to manually call connect and close after each connection, but that's easily gained by simply writing a context manager. I could get *some* limited benefit by raising the minconn value, but then I risk having connections that are *never* used, yet still taking resources on the DB server. Ideally, it would open as many connections as are needed, and then leave them open for future requests, perhaps with an "idle" timeout. Is there any way to achieve this behavior? ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From greg.ewing at canterbury.ac.nz Fri Jun 2 21:29:21 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 03 Jun 2017 13:29:21 +1200 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: Message-ID: Chris Angelico wrote: > Always using a context manager is good practice and > great for code clarity. Another thing about my Firebird interface was that you were forced to always use transactions, because the transaction object was the only thing that had methods for executing statements. -- Greg From greg.ewing at canterbury.ac.nz Fri Jun 2 21:46:24 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 03 Jun 2017 13:46:24 +1200 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: <4cdric1iau91j5p32mecn407ta8a0ad8g4@4ax.com> Message-ID: Chris Angelico wrote: > with psycopg2.connect(...) as conn: > with conn.trans() as trn: > for row in trn.execute("select ..."): > print(row) > > The outer context manager is optional, but not the inner one While I fully support making the use of transactions mandatory, I wouldn't like to be forced to use them in a with statement. In the application that I originally built my Firebird interface for, I had a mechanism where a user could open up a piece of data for editing, and then choose to save or cancel the edits. I implemented it by keeping a transaction around for the duration and then committing it or rolling it back. If a with statement were required around all transactions, I wouldn't have been able to do that. -- Greg From rosuav at gmail.com Fri Jun 2 23:56:26 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 3 Jun 2017 13:56:26 +1000 Subject: Python DB API - commit() v. execute("commit transaction")? In-Reply-To: References: <4cdric1iau91j5p32mecn407ta8a0ad8g4@4ax.com> Message-ID: On Sat, Jun 3, 2017 at 11:46 AM, Gregory Ewing wrote: > Chris Angelico wrote: >> >> with psycopg2.connect(...) as conn: >> with conn.trans() as trn: >> for row in trn.execute("select ..."): >> print(row) >> >> The outer context manager is optional, but not the inner one > > > While I fully support making the use of transactions mandatory, > I wouldn't like to be forced to use them in a with statement. > > In the application that I originally built my Firebird interface > for, I had a mechanism where a user could open up a piece of > data for editing, and then choose to save or cancel the edits. > I implemented it by keeping a transaction around for the > duration and then committing it or rolling it back. If a > with statement were required around all transactions, I > wouldn't have been able to do that. You wouldn't be FORCED to, but it would be strongly recommended. You could simply: trn = conn.trans() and then use it that way, but then you're responsible for calling trn.commit() or trn.rollback(). You would also be responsible for the longevity of your locks; if you hold a transaction waiting for a human, you potentially keep some things locked for a long time. Which is probably intentional as regards the primary record being edited, but you'd also hold locks on anything else you touch too. BTW, it should be possible to do: with trn.trans() as subtrn: on DBMSes that support subtransactions (eg PostgreSQL). For what that's worth. ChrisA From davidbenny2000 at gmail.com Sat Jun 3 07:00:12 2017 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Sat, 3 Jun 2017 04:00:12 -0700 (PDT) Subject: how to convert json to csv with python? Message-ID: <8e11ca54-7669-4018-ba44-7c770ddef5b0@googlegroups.com> i use https://github.com/evidens/json2csv Error: Traceback (most recent call last): File "json2csv.py", line 148, in loader.load(args.json_file) File "json2csv.py", line 53, in load self.process_each(json.load(json_file)) File "C:\Python27\lib\json\__init__.py", line 291, in load **kw) File "C:\Python27\lib\json\__init__.py", line 339, in loads return _default_decoder.decode(s) File "C:\Python27\lib\json\decoder.py", line 367, in decode raise ValueError(errmsg("Extra data", s, end, len(s))) ValueError: Extra data: line 10 column 2 - line 50 column 2 (char 224 - 1179) sample file is { "ip": "184.85.123.122", "hostname": "No Hostname", "city": "Cambridge", "region": "Massachusetts", "country": "US", "loc": "42.3626,-71.0843", "org": "AS20940 Akamai International B.V.", "postal": "02142" }, { "ip": "203.185.0.32", "hostname": "203185000032.ctinets.com", "city": "Central District", "region": "", "country": "HK", "loc": "22.2910,114.1500", "org": "AS9269 HKBN AS10103" }, { "ip": "184.85.123.122", "hostname": "a184-85-123-122.deploy.static.akamaitechnologies.com", "city": "Cambridge", "region": "Massachusetts", "country": "US", "loc": "42.3626,-71.0843", "org": "AS20940 Akamai International B.V.", "postal": "02142" }, { "ip": "203.185.0.32", "hostname": "No Hostname", "city": "Central District", "region": "", "country": "HK", "loc": "22.2910,114.1500", "org": "AS9269 HKBN AS10103", }, { "ip": "184.85.123.122", "hostname": "a184-85-123-122.deploy.static.akamaitechnologies.com", "city": "Cambridge", "region": "Massachusetts", "country": "US", "loc": "42.3626,-71.0843", "org": "AS20940 Akamai International B.V.", "postal": "02142" } outline is { "map": [["ip","ip"],["hostname", "hostname"],["city", "city"],["region", "region"],["country" ,"country"],["loc", "loc"],["org", "org"],["postal", "postal"]] } From kwpolska at gmail.com Sat Jun 3 07:20:04 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Sat, 3 Jun 2017 13:20:04 +0200 Subject: how to convert json to csv with python? In-Reply-To: <8e11ca54-7669-4018-ba44-7c770ddef5b0@googlegroups.com> References: <8e11ca54-7669-4018-ba44-7c770ddef5b0@googlegroups.com> Message-ID: On 3 June 2017 at 13:00, Ho Yeung Lee wrote: > i use > https://github.com/evidens/json2csv > > Error: > Traceback (most recent call last): > File "json2csv.py", line 148, in > loader.load(args.json_file) > File "json2csv.py", line 53, in load > self.process_each(json.load(json_file)) > File "C:\Python27\lib\json\__init__.py", line 291, in load > **kw) > File "C:\Python27\lib\json\__init__.py", line 339, in loads > return _default_decoder.decode(s) > File "C:\Python27\lib\json\decoder.py", line 367, in decode > raise ValueError(errmsg("Extra data", s, end, len(s))) > ValueError: Extra data: line 10 column 2 - line 50 column 2 (char 224 - 1179) > > sample file is > { > "ip": "184.85.123.122", > "hostname": "No Hostname", > "city": "Cambridge", > "region": "Massachusetts", > "country": "US", > "loc": "42.3626,-71.0843", > "org": "AS20940 Akamai International B.V.", > "postal": "02142" > }, > { > "ip": "203.185.0.32", > "hostname": "203185000032.ctinets.com", > "city": "Central District", > "region": "", > "country": "HK", > "loc": "22.2910,114.1500", > "org": "AS9269 HKBN AS10103" > }, > [snip] This is invalid JSON. You need to wrap all your dicts in a JSON array, like this: [ { "ip": "?" }, { "ip": "?" } ] (just add [ and ] to the start and end of your file) -- Chris Warrick PGP: 5EAAEA16 From davidbenny2000 at gmail.com Sat Jun 3 07:32:15 2017 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Sat, 3 Jun 2017 04:32:15 -0700 (PDT) Subject: how to convert json to csv with python? In-Reply-To: References: <8e11ca54-7669-4018-ba44-7c770ddef5b0@googlegroups.com> Message-ID: after edit the file, Traceback (most recent call last): File "json2csv.py", line 148, in loader.load(args.json_file) File "json2csv.py", line 53, in load self.process_each(json.load(json_file)) File "C:\Python27\lib\json\__init__.py", line 291, in load **kw) File "C:\Python27\lib\json\__init__.py", line 339, in loads return _default_decoder.decode(s) File "C:\Python27\lib\json\decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python27\lib\json\decoder.py", line 380, in raw_decode obj, end = self.scan_once(s, idx) ValueError: Expecting property name: line 38 column 1 (char 871) got another error Chris Warrick? 2017?6?3???? UTC+8??7?20?34???? > On 3 June 2017 at 13:00, Ho Yeung Lee wrote: > > i use > > https://github.com/evidens/json2csv > > > > Error: > > Traceback (most recent call last): > > File "json2csv.py", line 148, in > > loader.load(args.json_file) > > File "json2csv.py", line 53, in load > > self.process_each(json.load(json_file)) > > File "C:\Python27\lib\json\__init__.py", line 291, in load > > **kw) > > File "C:\Python27\lib\json\__init__.py", line 339, in loads > > return _default_decoder.decode(s) > > File "C:\Python27\lib\json\decoder.py", line 367, in decode > > raise ValueError(errmsg("Extra data", s, end, len(s))) > > ValueError: Extra data: line 10 column 2 - line 50 column 2 (char 224 - 1179) > > > > sample file is > > { > > "ip": "184.85.123.122", > > "hostname": "No Hostname", > > "city": "Cambridge", > > "region": "Massachusetts", > > "country": "US", > > "loc": "42.3626,-71.0843", > > "org": "AS20940 Akamai International B.V.", > > "postal": "02142" > > }, > > { > > "ip": "203.185.0.32", > > "hostname": "203185000032.ctinets.com", > > "city": "Central District", > > "region": "", > > "country": "HK", > > "loc": "22.2910,114.1500", > > "org": "AS9269 HKBN AS10103" > > }, > > [snip] > > This is invalid JSON. You need to wrap all your dicts in a JSON array, > like this: > > [ > { > "ip": "?" > }, > { > "ip": "?" > } > ] > > (just add [ and ] to the start and end of your file) > > -- > Chris Warrick > PGP: 5EAAEA16 From jon+usenet at unequivocal.eu Sat Jun 3 07:33:09 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Sat, 3 Jun 2017 11:33:09 -0000 (UTC) Subject: Python DB API - commit() v. execute("commit transaction")? References: Message-ID: On 2017-06-02, Chris Angelico wrote: > On Sat, Jun 3, 2017 at 5:31 AM, Jon Ribbens wrote: >> I'm not saying that transactional DDL is insane (it isn't), but MyISAM >> tables having transactions *only* for DDL is... surprising. Especially >> when it suddenly appeared as a "feature" in between two versions. It >> took me quite a while to work out why our database was randomly hanging. > > Wait, you have transactions with MyISAM now? I thought MySQL supported > transactions with InnoDB but not MyISAM, and the reason you didn't get > transactional DDL was that the system catalog tables are mandatorily > MyISAM, even if all your own tables are InnoDB. It's not so much as properly transactional as it is "the database wedges if a transaction is still open even though MyISAM tables aren't supposed to have transactions in the first place". The combination of "Python-MYSQLdb suddenly turned off autocommit" and "MySQL suddenly made transactions relevant to MyISAM" caused me a certain amount of difficulties a while back. From __peter__ at web.de Sat Jun 3 07:52:59 2017 From: __peter__ at web.de (Peter Otten) Date: Sat, 03 Jun 2017 13:52:59 +0200 Subject: how to convert json to csv with python? References: <8e11ca54-7669-4018-ba44-7c770ddef5b0@googlegroups.com> Message-ID: Ho Yeung Lee wrote: > after edit the file, > > Traceback (most recent call last): > File "json2csv.py", line 148, in > loader.load(args.json_file) > File "json2csv.py", line 53, in load > self.process_each(json.load(json_file)) > File "C:\Python27\lib\json\__init__.py", line 291, in load > **kw) > File "C:\Python27\lib\json\__init__.py", line 339, in loads > return _default_decoder.decode(s) > File "C:\Python27\lib\json\decoder.py", line 364, in decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "C:\Python27\lib\json\decoder.py", line 380, in raw_decode > obj, end = self.scan_once(s, idx) > ValueError: Expecting property name: line 38 column 1 (char 871) > > got another error So have a look at line 38 and see if you find something suspicious. { ... "org": "AS9269 HKBN AS10103", } Unlike Python dict literals json does not allow the trailing comma a the end of an {...} object. Compare: >>> json.loads('{"a": 1}') {'a': 1} >>> json.loads('{"a": 1,}') Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.4/json/__init__.py", line 318, in loads return _default_decoder.decode(s) File "/usr/lib/python3.4/json/decoder.py", line 343, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.4/json/decoder.py", line 359, in raw_decode obj, end = self.scan_once(s, idx) ValueError: Expecting property name enclosed in double quotes: line 1 column 9 (char 8) From chitturk at uah.edu Sat Jun 3 09:44:45 2017 From: chitturk at uah.edu (chitturk at uah.edu) Date: Sat, 3 Jun 2017 06:44:45 -0700 (PDT) Subject: Transitioning from Linux to Windows Message-ID: I am looking for suggestions, ideas. I have developed python (3.6.x, 2.7.x) scripts that run well as a user on an ubuntu/16.04 system - the scripts look for files, parses the files, assembles an output for the user. I first cd into a particular directory on the system (where I know the files exist) and run the script - the final result is in my working directory. What I am now trying to do is ... figure out the least painful way to have other users do the same - BUT sitting in front of a Windows desktop (7 or 10). Ideally, I would like to set up the user on their Windows 7/10 system so that they can "login" to the ubuntu system (say putty) - change working directory (to where desired) - run the script (on the ubuntu system) - and scp the file back to the windows desktop. ("porting" the ubuntu script to anaconda(3) on the windows desktop IS possible (but it has not been as easy as I had hoped!) (django and programs like that do seem to provide a "GUI" to have python scripts run on ubuntu/systems - but the setup looks mysterious/complicated (to me anyway)) I stumbled onto "paramiko" - is that a possible answer? Any suggestion/ideas would be greatly appreciated! (I am perfectly willing to stick to 3.6.x, unless there is a clean/easy solution using 2.7.x) krishnan From rosuav at gmail.com Sat Jun 3 09:50:12 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 3 Jun 2017 23:50:12 +1000 Subject: Transitioning from Linux to Windows In-Reply-To: References: Message-ID: On Sat, Jun 3, 2017 at 11:44 PM, wrote: > Ideally, I would like to set up the user on their Windows 7/10 system so that they can "login" to the ubuntu system (say putty) - change working directory (to where desired) - run the script (on the ubuntu system) - and scp the file back to the windows desktop. > > ("porting" the ubuntu script to anaconda(3) on the windows desktop IS possible (but it has not been as easy as I had hoped!) (django and programs like that do seem to provide a "GUI" to have python scripts run on ubuntu/systems - but the setup looks mysterious/complicated (to me anyway)) > Hmm. ISTM the easiest way would be to run an explicit server, probably HTTP (hence Django as you mentioned), and then you can have people access it using a client on Windows. With HTTP, that client would simply be a web browser. I would advise picking up a quick tutorial on Django or Flask, and seeing how easy it is to spin up an app and start responding to requests. ChrisA From chitturk at uah.edu Sat Jun 3 10:24:44 2017 From: chitturk at uah.edu (chitturk at uah.edu) Date: Sat, 3 Jun 2017 07:24:44 -0700 (PDT) Subject: Transitioning from Linux to Windows In-Reply-To: References: Message-ID: <15d6fae1-ce98-416c-90a6-dc850fed996e@googlegroups.com> On Saturday, June 3, 2017 at 8:50:27 AM UTC-5, Chris Angelico wrote: > Hmm. ISTM the easiest way would be to run an explicit server, probably > HTTP (hence Django as you mentioned), and then you can have people > access it using a client on Windows. With HTTP, that client would > simply be a web browser. I would advise picking up a quick tutorial on > Django or Flask, and seeing how easy it is to spin up an app and start > responding to requests. > > ChrisA Yes, "django" indeed seems to be a good solution - But I am having a tough time understanding how to set it up :( ... All I need is a simple example (say in django) with explicit directions "django for dummies" - that shows how to set it up - access is using http(s) and execute on the server/ubuntu ... I have indeed looked far and wide for the "simplest" example in django I can learn from/adapt ... it is something I have struggled with for sure ... (number crunching is what I do know, UI's are strange to me!) From barkerg at davincibb.net Sat Jun 3 10:42:14 2017 From: barkerg at davincibb.net (Gary Barker) Date: Sat, 3 Jun 2017 08:42:14 -0600 Subject: How to change variable from list to float Message-ID: <94a99949-8954-433d-a566-647aa36569ae@davincibb.net> I have searched for a solution to this but have not found a suitable example. The attached code generates this error: Traceback (most recent call last): File "calcsignal.py", line 7, in siglevfromexist = 34.8 + existattn TypeError: unsupported operand type(s) for +: 'float' and 'list' How do I convert the list variable (i.e. existattn) to a float? Operating details are: Python 3.4.2 Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux The following lines are the code in calcsignal.py: azdegpattrev = -47.40715077970316 azattndic = {-0.9: [-0.55], -0.5: [-0.46], 3.5: [-21.0], 1.4: [-7.48], 5.5: [-25.0], 0.0: [0.0], 1.9: [-21.0], 13.0: [-38.0], 15.0: [-39.0], 3.6: [-25.0], 20.0: [-39.0], -1.4: [-7.48], 90.0: [-65.0], -0.4: [-0.39], 0.5: [-0.46], 0.1: [-0.04], 1.0: [-1.31], -90.0: [-65.0], 40.0: [-42.0], 180.0: [-65.0], 1.5: [-10.0], -1.2: [-3.69], 0.3: [-0.28], -0.3: [-0.28], 0.2: [-0.15], -0.1: [-0.04], 1.1: [-2.34], -180.0: [-65.0], -0.2: [-0.15], 1.2: [-3.69], -40.0: [-42.0], 0.4: [-0.39], -5.5: [-25.0], -1.5: [-10.0], -20.0: [-39.0], 0.9: [-0.55], -3.5: [-21.0], -1.9: [-21.0], -15.0: [-39.0], -13.0: [-38.0], 1.3: [-5.39], -1.3: [-5.39], -3.6: [-25.0], -1.0: [-1.31], -1.1: [-2.34]} azlist = [-0.9, -0.5, 3.5, 1.4, 5.5, 0.0, 1.9, 13.0, 15.0, 3.6, 20.0, -1.4, 90.0, -0.4, 0.5, 0.1, 1.0, -90.0, 40.0, 180.0, 1.5, -1.2, 0.3, -0.3, 0.2, -0.1, 1.1, -180.0, -0.2, 1.2, -40.0, 0.4, -5.5, -1.5, -20.0, 0.9, -3.5, -1.9, -15.0, -13.0, 1.3, -1.3, -3.6, -1.0, -1.1] azlist = [float(i) for i in azlist] closestaz = min(azlist, key=lambda x: abs(x - azdegpattrev)) existattn = azattndic[closestaz] siglevfromexist = 34.8 + existattn From walters.justin01 at gmail.com Sat Jun 3 10:50:08 2017 From: walters.justin01 at gmail.com (justin walters) Date: Sat, 3 Jun 2017 07:50:08 -0700 Subject: Transitioning from Linux to Windows In-Reply-To: References: <15d6fae1-ce98-416c-90a6-dc850fed996e@googlegroups.com> Message-ID: On Jun 3, 2017 7:28 AM, wrote: On Saturday, June 3, 2017 at 8:50:27 AM UTC-5, Chris Angelico wrote: > Hmm. ISTM the easiest way would be to run an explicit server, probably > HTTP (hence Django as you mentioned), and then you can have people > access it using a client on Windows. With HTTP, that client would > simply be a web browser. I would advise picking up a quick tutorial on > Django or Flask, and seeing how easy it is to spin up an app and start > responding to requests. > > ChrisA Yes, "django" indeed seems to be a good solution - But I am having a tough time understanding how to set it up :( ... All I need is a simple example (say in django) with explicit directions "django for dummies" - that shows how to set it up - access is using http(s) and execute on the server/ubuntu ... I have indeed looked far and wide for the "simplest" example in django I can learn from/adapt ... it is something I have struggled with for sure ... (number crunching is what I do know, UI's are strange to me!) -- https://mail.python.org/mailman/listinfo/python-list Lucky for you! I work with django almost every single day! The best place to get started is the official tutorial. https://docs.djangoproject.com/en/1.11/intro/tutorial01/ Or should take a couple of hours to get through it, but it will teach you all the basics. After that, you want to learn how to set up gunicorn to serve a local unix socket. Then you need to set up nginx to proxy all requests to that socket. You can use certbot with letsencrypt dor ssl. Alternatively, i believe heroku has a prebuilt django deployment you can use. From skyteacherusers at gmail.com Sat Jun 3 10:53:06 2017 From: skyteacherusers at gmail.com (skyteacherusers at gmail.com) Date: Sat, 3 Jun 2017 07:53:06 -0700 (PDT) Subject: noCaptcha cracking handler Message-ID: <68070780-64d3-401a-a7e0-ee2092ec62b8@googlegroups.com> Hey, Please check out my latest repository https://github.com/gubrul/noCaptcha Thanks:) From no.email at nospam.invalid Sat Jun 3 14:00:08 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 03 Jun 2017 11:00:08 -0700 Subject: [OT] How to improve my programming skills? References: <5930321B.9040405@googlemail.com> Message-ID: <87a85pne8n.fsf@nightsong.com> Mirko writes: > TLDR: Sorry for OT. Long-time Linux geek and hobby programmer wants to > improve his coding skills. What's most important: project planing, > algorithms and data structures, contributing to FOSS, web development, > learning other languages or something else? If it's specifically about coding skills, the most important thing is reading and writing lots of code. That said, coding skills are just one facet of a good developer. The most important developers in the group I work with aren't necessarily the strongest coders. They're the ones with the best knowledge of the (large) program's organization, how to use the testing frameworks (CI system), and above all Git ;-). One coding exercise I like is to write some simple program and then tweak the code until you think it is perfect. That's similar to improving your writing by editing something you've written til you get all the phrasing just right. > - I never sit down and plan anything with pencil and paper, > flowcharts, UML or anything else. That's not worthwhile unless you're doing some very complicated, and even then it's usually just some diagrams and some written docs. Flowcharts were a 1950s thing and UML is a corporate Java thing. > - I never learned algorithms and data structures. I know *what* > (linked) lists, dicts, arrays, structs, and trees are; what binary > search or bubble-sort is, but I never really studied them, let alone > implemented them for educative purposes. I'd say Python saves you from having to understand how stuff like linked lists and hash tables work, at least at first. It will become important but not for now. If you were programming in C it would all be important and fundamental immediately. But, read this: http://antirez.com/news/112 > - When it comes to coding, I'm heavily shy and unsure. Do you play a musical instrument? Think of how much more confident you became with practice, even if you never got very good at it. > But web programming always feel like being trapped in a > mangrove jungle, Yeah, I feel the same way and avoid that stuff. > - I'm very hard to motivate, when the issue or topic doesn't interest > me much. I know some tricks to increase my motivation in such cases, > but don't use them enough. For doing stuff on your own, it's fine to narrow them down to stuff that excites you and that you want to keep doing. It does help to cultivate persistence for things like debugging. From no.email at nospam.invalid Sat Jun 3 14:03:25 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 03 Jun 2017 11:03:25 -0700 Subject: noCaptcha cracking handler References: <68070780-64d3-401a-a7e0-ee2092ec62b8@googlegroups.com> Message-ID: <8760gdne36.fsf@nightsong.com> skyteacherusers at gmail.com writes: > Please check out my latest repository > https://github.com/gubrul/noCaptcha Thanks. Captchas have gotten to be a real PITA lately, even as someone who doesn't want to run automated clients on the infected sites. I remember getting sick of trying to identify pictures of buildings with store fronts, and thinking maybe there should be a bot to do them... oh wait... From rosuav at gmail.com Sat Jun 3 15:01:23 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 4 Jun 2017 05:01:23 +1000 Subject: Transitioning from Linux to Windows In-Reply-To: <15d6fae1-ce98-416c-90a6-dc850fed996e@googlegroups.com> References: <15d6fae1-ce98-416c-90a6-dc850fed996e@googlegroups.com> Message-ID: On Sun, Jun 4, 2017 at 12:24 AM, wrote: > On Saturday, June 3, 2017 at 8:50:27 AM UTC-5, Chris Angelico wrote: > >> Hmm. ISTM the easiest way would be to run an explicit server, probably >> HTTP (hence Django as you mentioned), and then you can have people >> access it using a client on Windows. With HTTP, that client would >> simply be a web browser. I would advise picking up a quick tutorial on >> Django or Flask, and seeing how easy it is to spin up an app and start >> responding to requests. >> >> ChrisA > > Yes, "django" indeed seems to be a good solution - But I am having a tough time understanding how to set it up :( ... > > All I need is a simple example (say in django) with explicit directions "django for dummies" - that shows how to set it up - access is using http(s) and execute on the server/ubuntu ... I have indeed looked far and wide for the "simplest" example in django I can learn from/adapt ... it is something I have struggled with for sure ... (number crunching is what I do know, UI's are strange to me!) There are some great Django tutorials out there; I believe the Django Girls tutorial is a great one (whether you're a girl or not): https://tutorial.djangogirls.org/en/ Alternatively, Flask tends to be simpler to set up than Django is. Here's a simple app I built a while ago that uses Flask: https://github.com/Rosuav/Flask1 As you can see, all the code resides in a single file. (That's not how I would structure a full-size app, but for what you're doing here, it would be fine.) You don't need to worry about WSGI or anything, just run your program directly; when it hits app.run() down the bottom, it'll process requests until you halt it with Ctrl-C. The Web is an important part of today's world, and time spent getting familiar with the use of a web browser as your UI is time well spent. You should be able to master this in a weekend, I would guess. ChrisA From nobozo at gmail.com Sat Jun 3 15:10:40 2017 From: nobozo at gmail.com (Jon Forrest) Date: Sat, 3 Jun 2017 12:10:40 -0700 Subject: Is An Element of a Sequence an Object? Message-ID: I'm learning about Python. A book I'm reading about it says "... a string in Python is a sequence. A sequence is an ordered collection of objects". This implies that each character in a string is itself an object. This doesn't seem right to me, but since I'm just learning Python I questioned the author about this. He gave an example the displays the ids of string slices. These ids are all different, but I think that's because the slicing operation creates objects. I'd like to suggest an explanation of what a sequence is that doesn't use the word 'object' because an object has a specific meaning in Python. Am I on the right track here? Cordially, Jon Forrest From tjol at tjol.eu Sat Jun 3 15:38:03 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 3 Jun 2017 21:38:03 +0200 Subject: Is An Element of a Sequence an Object? In-Reply-To: References: Message-ID: <5933103c$0$1775$e4fe514c@news.kpn.nl> On 03/06/17 21:10, Jon Forrest wrote: > I'm learning about Python. A book I'm reading about it > says "... a string in Python is a sequence. A sequence is an ordered > collection of objects". This implies that each character in a string > is itself an object. > > This doesn't seem right to me, but since I'm just learning Python > I questioned the author about this. He gave an example the displays > the ids of string slices. These ids are all different, but I think > that's because the slicing operation creates objects. > > I'd like to suggest an explanation of what a sequence is > that doesn't use the word 'object' because an object has > a specific meaning in Python. > > Am I on the right track here? No, strings don't internally store the characters as objects, and yes, the slicing operation creates objects. However, strings *are* sequences, sequences *are* ordered collections of objects. The sentence "A sequence is an ordered collection of objects" means that a sequence has (ordered) elements, you can access these elements, and when you do that, you will get an object (seeing as everything is an object). It does *not* imply that all the elements of a sequence exist in memory before you access them. This definition of a sequence describes the behaviour, not the implementation, of a sequence object. Another built-in sequence type that doesn't bother to store all of its elements as objects, or at all, is range: >>> r = range(1000000000) # Does NOT create one billion objects >>> i = r[10000] # The element is an object. >>> i 10000 >>> id(i) 139758670321552 >>> i is 10000 False >>> NB: in Python 2, range does store all of its elements as objects. Running this code in Python 2 will consume many gigabytes of memory. If you have to use Python 2, use xrange instead. Hope this helps. Thomas From wolfgang.maier at biologie.uni-freiburg.de Sat Jun 3 15:48:45 2017 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Sat, 3 Jun 2017 21:48:45 +0200 Subject: Transitioning from Linux to Windows In-Reply-To: References: Message-ID: On 03.06.2017 15:44, chitturk at uah.edu wrote: > I am looking for suggestions, ideas. > > I have developed python (3.6.x, 2.7.x) scripts that run well as a user on an ubuntu/16.04 system - the scripts look for files, parses the files, assembles an output for the user. > > I first cd into a particular directory on the system (where I know the files exist) and run the script - the final result is in my working directory. > > What I am now trying to do is ... figure out the least painful way to have other users do the same - BUT sitting in front of a Windows desktop (7 or 10). > > Ideally, I would like to set up the user on their Windows 7/10 system so that they can "login" to the ubuntu system (say putty) - change working directory (to where desired) - run the script (on the ubuntu system) - and scp the file back to the windows desktop. > > ("porting" the ubuntu script to anaconda(3) on the windows desktop IS possible (but it has not been as easy as I had hoped!) (django and programs like that do seem to provide a "GUI" to have python scripts run on ubuntu/systems - but the setup looks mysterious/complicated (to me anyway)) > > I stumbled onto "paramiko" - is that a possible answer? > > Any suggestion/ideas would be greatly appreciated! > > (I am perfectly willing to stick to 3.6.x, unless there is a clean/easy > solution using 2.7.x) > > krishnan > An alternative to writing your own server app using django, though admittedly less exciting from a developer's point of view, could be x2goclient/server (http://wiki.x2go.org/doku.php/start). That would be more like your putty suggestion, but a lot more user-friendly. Wolfgang From mbg1708 at planetmail.com Sat Jun 3 16:35:33 2017 From: mbg1708 at planetmail.com (mbg1708 at planetmail.com) Date: Sat, 3 Jun 2017 13:35:33 -0700 (PDT) Subject: [OT] How to improve my programming skills? In-Reply-To: References: <5930321B.9040405@googlemail.com> Message-ID: <7fd66a10-16bd-4b7c-be70-f372d8c236f3@googlegroups.com> > I'm looking for a way (*the* way, ie. the "BEST(tm)" way) to improve > my coding skills. While I'm a quite hard-core computer geek since 25 > years and a really good (hobbyist) Linux-SOHO-Admin, my programming > skills are less than sub-par. I wish to change that and become at > least am average hobby-programmer. But what do you want to DO with these new improved programming skills? Taking a little time to answer this question will help. For example, suppose that you want to reduce the time you spend using your Linux-SOHO-Admin skills, then perhaps you might learn more about the architecture of Fedora (or Debian), and then learn more about bash and C. Or, in another example, you may want to write a GUI application to improve or simplify some process which you have written and which you believe could be made better. This might lead you to learn Glade and Python3. But after the LEARNING, you need to scope out a specific, probably small, PROJECT which you might complete in, say, one month. Write short requirements (elsewhere someone suggested use cases -- good suggestion). If a GUI is involved, sketch out screen mock ups (pencil and paper). Determine if the requirements fall naturally into separate parts -- this might be an early clue about the possible modularity of your project. Then start programming. If you have found possible modularity, then test each module as you go. Finally, in my experience, the first time round is a huge learning experience. So go back and re-write from scratch. The second product will almost certainly be of higher quality than the first, and you might find that the second go round would be better written using different tools.....more learning....and so on. From rossiheron at gmail.com Sat Jun 3 17:17:49 2017 From: rossiheron at gmail.com (Heron Rossi) Date: Sat, 3 Jun 2017 14:17:49 -0700 (PDT) Subject: Are celery threads independent? Message-ID: Are they independent in a way that each thread will spawn a new process and won't share resources? From sean.dizazzo at gmail.com Sat Jun 3 17:59:01 2017 From: sean.dizazzo at gmail.com (Sean DiZazzo) Date: Sat, 3 Jun 2017 14:59:01 -0700 (PDT) Subject: Bug or intended behavior? In-Reply-To: References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <5d0eaeff-55b1-5989-df92-6f3c2f6a33e7@gmail.com> Message-ID: <3c3d3d9f-9ee9-4610-a232-2d0565dcf061@googlegroups.com> On Friday, June 2, 2017 at 10:46:03 AM UTC-7, bob gailer wrote: > On 6/2/2017 1:28 PM, Jussi Piitulainen wrote: > > sean.dizazzo at gmail.com writes: > > > >> Can someone please explain this to me? Thanks in advance! > >> > >> ~Sean > >> > >> > >> Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) > >> [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin > >> Type "help", "copyright", "credits" or "license" for more information. > >>>>> print "foo %s" % 1-2 > >> Traceback (most recent call last): > >> File "", line 1, in > >> TypeError: unsupported operand type(s) for -: 'str' and 'int' > > The per cent operator has precedence over minus. Spacing is not > > relevant. Use parentheses. > > > In other words "foo %s" % 1 is executed, giving "1". Then "1"-2 is > attempted giving the error. > Also: If there is more than one conversion specifier the right argument > to % must be a tuple. > I usually write a tuple even if there is only one conversion specifier - > that avoids the problem > you encountered and makes it easy to add more values when you add more > conversion specifiers. > > print "foo %s" % (1-2,) > > Bob Gailer I get what it's doing, it just doesn't make much sense to me. Looking at operator precedence, I only see the % operator in regards to modulus. Nothing in regards to string formatting. Is it just a side effect of the % being overloaded in strings? Or is it intentional that it's higher precedence...and why? Maybe I'm making too big a deal of it. It just doesn't 'feel' right to me. ~Sean From rosuav at gmail.com Sat Jun 3 18:05:54 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 4 Jun 2017 08:05:54 +1000 Subject: Bug or intended behavior? In-Reply-To: <3c3d3d9f-9ee9-4610-a232-2d0565dcf061@googlegroups.com> References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <5d0eaeff-55b1-5989-df92-6f3c2f6a33e7@gmail.com> <3c3d3d9f-9ee9-4610-a232-2d0565dcf061@googlegroups.com> Message-ID: On Sun, Jun 4, 2017 at 7:59 AM, Sean DiZazzo wrote: > I get what it's doing, it just doesn't make much sense to me. Looking at operator precedence, I only see the % operator in regards to modulus. Nothing in regards to string formatting. Is it just a side effect of the % being overloaded in strings? Or is it intentional that it's higher precedence...and why? > > Maybe I'm making too big a deal of it. It just doesn't 'feel' right to me. The operator is what it is regardless of the values on either side of it. The somewhat cute use of a division operator for either percent formatting or path joining doesn't change this. ChrisA From nobozo at gmail.com Sat Jun 3 19:42:49 2017 From: nobozo at gmail.com (Jon Forrest) Date: Sat, 3 Jun 2017 16:42:49 -0700 Subject: Is An Element of a Sequence an Object? In-Reply-To: <5933103c$0$1775$e4fe514c@news.kpn.nl> References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: On 6/3/2017 12:38 PM, Thomas Jollans wrote: >> I'd like to suggest an explanation of what a sequence is >> that doesn't use the word 'object' because an object has >> a specific meaning in Python. >> >> Am I on the right track here? > > No, strings don't internally store the characters as objects, and yes, > the slicing operation creates objects. However, strings *are* sequences, > sequences *are* ordered collections of objects. I don't see how both can be true. "Object" has a clear meaning in Python, and the contents of a sequence don't meet the requirements, as I understand them. If there were some way of measuring the number of objects in a program, then it would be easy to prove (or disprove) my hypothesis. In other words this program a = "abc" would have the same number of objects as this program a = "abcdefghijklmnopqrstuvwzyz" > The sentence "A sequence is an ordered collection of objects" means that > a sequence has (ordered) elements, you can access these elements, and > when you do that, you will get an object (seeing as everything is an > object). The distinction between an "object" and "element" is key here. (This might be seen as pedantic, but I think it's important to be clear, especially in a book intended for beginners, as I am. > Hope this helps. Thanks for taking the time to reply. I'm somebody who reads technical books *very* carefully so the distinction between objects and elements is important to me. Cordially, Jon Forrest From rosuav at gmail.com Sat Jun 3 19:58:23 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 4 Jun 2017 09:58:23 +1000 Subject: Is An Element of a Sequence an Object? In-Reply-To: References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: On Sun, Jun 4, 2017 at 9:42 AM, Jon Forrest wrote: > On 6/3/2017 12:38 PM, Thomas Jollans wrote: > >>> I'd like to suggest an explanation of what a sequence is >>> that doesn't use the word 'object' because an object has >>> a specific meaning in Python. >>> >>> Am I on the right track here? >> >> No, strings don't internally store the characters as objects, and yes, >> the slicing operation creates objects. However, strings *are* sequences, >> sequences *are* ordered collections of objects. > > I don't see how both can be true. "Object" has a clear meaning in > Python, and the contents of a sequence don't meet the requirements, > as I understand them. > > If there were some way of measuring the number of objects in > a program, then it would be easy to prove (or disprove) my hypothesis. > In other words this program > > a = "abc" > > would have the same number of objects as this program > > a = "abcdefghijklmnopqrstuvwzyz" A sequence doesn't necessarily "contain" anything. As has been mentioned, a range object is a sequence, but it creates integer objects lazily. The point of a sequence is that you can do this: for thing in sequence: and whatever is in 'thing' must, by definition, be an object. Since you can do this with a string, a string is, indeed, a sequence. (Technically there's more to a sequence than just being iterable, but this is the part that matters here.) The semantics of Python would be the same if all those objects were indeed pre-created, but for memory efficiency, they aren't. ChrisA From ben+python at benfinney.id.au Sat Jun 3 20:03:11 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 04 Jun 2017 10:03:11 +1000 Subject: Is An Element of a Sequence an Object? References: Message-ID: <85inkcy5z4.fsf@benfinney.id.au> Jon Forrest writes: > I'm learning about Python. A book I'm reading about it Can you say which book, and where in the book it says this? > says "... a string in Python is a sequence. A sequence is an ordered > collection of objects". This implies that each character in a string > is itself an object. It does imply that. To phrase it that way might be slightly misleading, though. It is *not* true, for example, to say that the characters in the string already exist as objects. I would prefer to state the implication this way: *When* treating the string as a sequence, it *produces* each character as a separate (length 1) string object. That is, the one-character string objects are generated when needed, and do not exist otherwise. > This doesn't seem right to me, but since I'm just learning Python > I questioned the author about this. He gave an example the displays > the ids of string slices. These ids are all different, but I think > that's because the slicing operation creates objects. That's a separate matter :-) A slice is itself an object, which allows dealing with part of a sequence. > I'd like to suggest an explanation of what a sequence is that doesn't > use the word 'object' because an object has a specific meaning in > Python. The term ?object? is correct in everything you've presented so far; I think that term is fine for what you're saying. I think, rather, that you may have an incorrect understanding of a sequence. The objects produced by an operation on a sequence ? a slice, or iterating the sequence ? do not necessarily exist as distinct objects before that operation. > Am I on the right track here? I hope that helps. -- \ ?The old believe everything; the middle-aged suspect | `\ everything; the young know everything.? ?Oscar Wilde, _Phrases | _o__) and Philosophies for the Use of the Young_, 1894 | Ben Finney From steve+python at pearwood.info Sat Jun 3 20:23:49 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 04 Jun 2017 10:23:49 +1000 Subject: Is An Element of a Sequence an Object? References: Message-ID: <59335318$0$1589$c3e8da3$5496439d@news.astraweb.com> On Sun, 4 Jun 2017 05:10 am, Jon Forrest wrote: > I'm learning about Python. A book I'm reading about it > says "... a string in Python is a sequence. A sequence is an ordered > collection of objects". This implies that each character in a string > is itself an object. Which book is this? Because its wrong. A sequence in Python is: An iterable which supports efficient element access using integer indices via the __getitem__() special method and defines a __len__() method that returns the length of the sequence. Some built-in sequence types are list, str, tuple, and bytes. Note that dict also supports __getitem__() and __len__(), but is considered a mapping rather than a sequence because the lookups use arbitrary immutable keys rather than integers. The collections.abc.Sequence abstract base class defines a much richer interface that goes beyond just __getitem__() and __len__(), adding count(), index(), __contains__(), and __reversed__(). Types that implement this expanded interface can be registered explicitly using register(). https://docs.python.org/3/glossary.html#term-sequence One way for an object to provide efficient element access is to store the elements as objects, e.g. what lists and tuples do. Another way is to calculate them on the fly, as range (xrange in Python 2) does. A third way is to copy the elements out of some other data structure as needed, as strings do, and array.array. We can fix the book's statement by changing it to: A sequence is an ordered collection of *elements* ... without specifying how the elements are stored (as Python objects, or as values in some low-level data structure, or even virtual elements which are computed on the fly as needed). > This doesn't seem right to me, but since I'm just learning Python > I questioned the author about this. He gave an example the displays > the ids of string slices. These ids are all different, but I think > that's because the slicing operation creates objects. It does indeed. > I'd like to suggest an explanation of what a sequence is > that doesn't use the word 'object' because an object has > a specific meaning in Python. > > Am I on the right track here? Absolutely. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From python at lucidity.plus.com Sat Jun 3 20:57:45 2017 From: python at lucidity.plus.com (Erik) Date: Sun, 4 Jun 2017 01:57:45 +0100 Subject: Is An Element of a Sequence an Object? In-Reply-To: References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: <85bc91bd-515c-83b7-62cd-8dee2afb998c@lucidity.plus.com> On 04/06/17 00:42, Jon Forrest wrote: > On 6/3/2017 12:38 PM, Thomas Jollans wrote: > > >> I'd like to suggest an explanation of what a sequence is > >> that doesn't use the word 'object' because an object has > >> a specific meaning in Python. > >> > >> Am I on the right track here? > > > > No, strings don't internally store the characters as objects, and yes, > > the slicing operation creates objects. However, strings *are* sequences, > > sequences *are* ordered collections of objects. > > I don't see how both can be true. "Object" has a clear meaning in > Python, and the contents of a sequence don't meet the requirements, > as I understand them. > > If there were some way of measuring the number of objects in > a program, then it would be easy to prove (or disprove) my hypothesis. > In other words this program You are confusing how a sequence might be implemented with how it can be thought of conceptually (*). I guess the book is explaining the conceptual model (which, as a book for learners, seems quite reasonable). Whenever you extract an element from a sequence (whether that is by iteration - such as "for item in seq:" - or by indexing - such as "seq[x]") - you receive an object. That doesn't mean that the sequence holds all of its elements physically in RAM as separate objects. It is up to the sequence (which is also an object ;)) to decide how to implement iteration and indexing. It might do that by creating and storing a whole lot of real objects in RAM or it might do it by remembering the _formula_ for the sequence and creating those objects as and when they are asked for. A list is an example of the first type (the list class doesn't know what heterogeneous objects might be appended to an instance, so it has to store an array of objects somehow). The object returned by range() (xrange() in Py2) is an example of the latter - the "sequence" is actually just a clever object which knows how to calculate and create efficiently an object for one of its elements when you ask for it. Hope that helps, E. (*) And that's great - you're actually thinking about this harder than a lot of people would. For a lot of learners the _conceptual_ model is good enough and that's all they'll ever need to understand. You are curious about the _implementation_ and that's good because now you'll see much more easily later how to program efficient sequence classes. From grant.b.edwards at gmail.com Sat Jun 3 22:15:33 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 4 Jun 2017 02:15:33 +0000 (UTC) Subject: Is An Element of a Sequence an Object? References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: On 2017-06-03, Thomas Jollans wrote: > On 03/06/17 21:10, Jon Forrest wrote: > >> I'm learning about Python. A book I'm reading about it >> says "... a string in Python is a sequence. A sequence is an ordered >> collection of objects". This implies that each character in a string >> is itself an object. You can think about it that way if you want, and from observable behavior you can't tell whether or not it's true. >> This doesn't seem right to me, but since I'm just learning Python I >> questioned the author about this. He gave an example the displays the >> ids of string slices. These ids are all different, but I think that's >> because the slicing operation creates objects. That doesn't belie the assumption that inside strings, characters are stored in memory as individual objects. Strings are immutable, so indexing and slicing can produce new objects or not -- it doesn't matter. >> I'd like to suggest an explanation of what a sequence is that doesn't >> use the word 'object' because an object has a specific meaning in >> Python. >> >> Am I on the right track here? > > No, strings don't internally store the characters as objects, Not in CPython, they don't. In some other (hypothetical) implementation, they could be. The memory usage speed implications of such a decision are not pleasant to contemplate. -- Grant From nobozo at gmail.com Sun Jun 4 00:06:19 2017 From: nobozo at gmail.com (Jon Forrest) Date: Sat, 3 Jun 2017 21:06:19 -0700 Subject: Is An Element of a Sequence an Object? In-Reply-To: <85inkcy5z4.fsf@benfinney.id.au> References: <85inkcy5z4.fsf@benfinney.id.au> Message-ID: On 6/3/2017 5:03 PM, Ben Finney wrote: > Jon Forrest writes: > >> I'm learning about Python. A book I'm reading about it > > Can you say which book, and where in the book it says this? With all due respect, I'd rather not. The author has been very responsive when I raised this issue, and I don't want to make him look bad. > To phrase it that way might be slightly misleading, though. It is *not* > true, for example, to say that the characters in the string already > exist as objects. That's exactly what I was thinking, and why I raised the issue. > I think, rather, that you may have an incorrect understanding of a > sequence. The objects produced by an operation on a sequence ? a slice, > or iterating the sequence ? do not necessarily exist as distinct objects > before that operation. I wasn't thinking so much about the objects produced by an operation on a sequence as I was about when the sequence is created. A trivial sequence is "abc" As I understand it, this is one object, not three. The original excerpt implies the later. Jon From nobozo at gmail.com Sun Jun 4 00:17:55 2017 From: nobozo at gmail.com (Jon Forrest) Date: Sat, 3 Jun 2017 21:17:55 -0700 Subject: Is An Element of a Sequence an Object? In-Reply-To: References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: On 6/3/2017 4:58 PM, Chris Angelico wrote: > A sequence doesn't necessarily "contain" anything. Maybe not always, but doesn't "abc" contain three characters? Is 'contain' the right word? > As has been mentioned, a range object is a sequence, but it creates integer > objects lazily. So there must be a class of sequence, like a literal string, that contains multiple characters stored as one object, and another class, like range(500000), that doesn't store anything, and only produces values when it's iterated over. Plus, a list is a sequence, but in this case it's clear that each element of a list is an object. Jon From nobozo at gmail.com Sun Jun 4 00:19:53 2017 From: nobozo at gmail.com (Jon Forrest) Date: Sat, 3 Jun 2017 21:19:53 -0700 Subject: Is An Element of a Sequence an Object? In-Reply-To: <59335318$0$1589$c3e8da3$5496439d@news.astraweb.com> References: <59335318$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 6/3/2017 5:23 PM, Steve D'Aprano wrote: > On Sun, 4 Jun 2017 05:10 am, Jon Forrest wrote: > We can fix the book's statement by changing it to: > > A sequence is an ordered collection of *elements* ... That's exactly what I was thinking, but then there'd have to be a clear definition of "element". Jon From steve at pearwood.info Sun Jun 4 00:31:17 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 04 Jun 2017 04:31:17 GMT Subject: Is An Element of a Sequence an Object? References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: <59338d14$0$2863$c3e8da3$76491128@news.astraweb.com> On Sun, 04 Jun 2017 02:15:33 +0000, Grant Edwards wrote: > On 2017-06-03, Thomas Jollans wrote: >> On 03/06/17 21:10, Jon Forrest wrote: >> >>> I'm learning about Python. A book I'm reading about it says "... a >>> string in Python is a sequence. A sequence is an ordered collection of >>> objects". This implies that each character in a string is itself an >>> object. > > You can think about it that way if you want, and from observable > behavior you can't tell whether or not it's true. Actually you can, and you recognise that yourself: [...] >> No, strings don't internally store the characters as objects, > > Not in CPython, they don't. In some other (hypothetical) > implementation, they could be. The memory usage speed implications of > such a decision are not pleasant to contemplate. Python strings would use a lot more memory if they were implemented in the way the mystery book suggests they are (each character being represented as a distinct object). In Python 3, for example: >>> import sys >>> sys.getsizeof("abcde") # actual memory consumption 54 >>> sum(sys.getsizeof(c) for c in "acbde") # theoretical 250 So we can tell the two implementations apart. -- Steve From steve at pearwood.info Sun Jun 4 00:33:22 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 04 Jun 2017 04:33:22 GMT Subject: Is An Element of a Sequence an Object? References: <85inkcy5z4.fsf@benfinney.id.au> Message-ID: <59338d92$0$2863$c3e8da3$76491128@news.astraweb.com> On Sat, 03 Jun 2017 21:06:19 -0700, Jon Forrest wrote: > I wasn't thinking so much about the objects produced by an operation on > a sequence as I was about when the sequence is created. A trivial > sequence is > > "abc" > > As I understand it, this is one object, not three. The original excerpt > implies the later. Actually it implies *four* objects, not three. A better example would be a tuple: ("a", "b", "c") which consists of the three items (each a single character string) plus the tuple object itself, the container holding them. -- Steve From rosuav at gmail.com Sun Jun 4 00:39:21 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 4 Jun 2017 14:39:21 +1000 Subject: Is An Element of a Sequence an Object? In-Reply-To: References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: On Sun, Jun 4, 2017 at 2:17 PM, Jon Forrest wrote: > On 6/3/2017 4:58 PM, Chris Angelico wrote: > >> A sequence doesn't necessarily "contain" anything. > > > Maybe not always, but doesn't > > "abc" > > contain three characters? Is 'contain' the right word? That's a tricky thing to pin down. Since it's possible for a sequence to contain itself, or to contain something which contains it, "contain" is clearly not in any physically-equivalent sense. Does the list [1, 2, 3] contain the float value 2.0? Not in any object-identity sense, and not in the sense that you'll get 2.0 by iterating over it, but yes, in the sense of __contains__. So this list "contains", in one sense, objects that may not have existed previously. In the same way, the string "abc" contains three characters, but it may not contain any objects representing them. In an abstract sense, the string DOES contain those characters. So long as we don't discuss the concrete definitions of objects, yes, we're fine. ChrisA From __peter__ at web.de Sun Jun 4 02:50:04 2017 From: __peter__ at web.de (Peter Otten) Date: Sun, 04 Jun 2017 08:50:04 +0200 Subject: Is An Element of a Sequence an Object? References: <5933103c$0$1775$e4fe514c@news.kpn.nl> <59338d14$0$2863$c3e8da3$76491128@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Sun, 04 Jun 2017 02:15:33 +0000, Grant Edwards wrote: > >> On 2017-06-03, Thomas Jollans wrote: >>> On 03/06/17 21:10, Jon Forrest wrote: >>> >>>> I'm learning about Python. A book I'm reading about it says "... a >>>> string in Python is a sequence. A sequence is an ordered collection of >>>> objects". This implies that each character in a string is itself an >>>> object. >> >> You can think about it that way if you want, and from observable >> behavior you can't tell whether or not it's true. > > Actually you can, and you recognise that yourself: > > [...] >>> No, strings don't internally store the characters as objects, >> >> Not in CPython, they don't. In some other (hypothetical) >> implementation, they could be. The memory usage speed implications of >> such a decision are not pleasant to contemplate. > > Python strings would use a lot more memory if they were implemented in > the way the mystery book suggests they are (each character being > represented as a distinct object). > > In Python 3, for example: > > >>>> import sys >>>> sys.getsizeof("abcde") # actual memory consumption > 54 >>>> sum(sys.getsizeof(c) for c in "acbde") # theoretical > 250 > > > So we can tell the two implementations apart. I str were implemented as a sequence of character objects it would still report >>> sys.getsizeof(tuple("abcde")) 88 The tuple doesn't store objects either, it holds pointers to objects and could easily be changed to hold the values of small integers, say. From marko at pacujo.net Sun Jun 4 03:03:03 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 04 Jun 2017 10:03:03 +0300 Subject: Is An Element of a Sequence an Object? References: <59335318$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: <878tl8uteg.fsf@elektro.pacujo.net> Jon Forrest : > On 6/3/2017 5:23 PM, Steve D'Aprano wrote: >> On Sun, 4 Jun 2017 05:10 am, Jon Forrest wrote: > >> We can fix the book's statement by changing it to: >> >> A sequence is an ordered collection of *elements* ... > > That's exactly what I was thinking, but then there'd have to > be a clear definition of "element". Instead of getting into metaphysical explanations, one should define the concepts operationally. A *sequence* is an object s that supports (most of) these operations: x in s x not in s s + t s * n n * s s[i] s[i:j] s[i:j:k] len(s) min(s) max(s) s.index(x[, i[, j]]) s.count(x) As sequence is *mutable* if it additionally supports (most of) these operations: x[i] = x x[i:j] = t del s[i:j] s[i:j:k] = t del s[i:j:k] s.append(x) s.clear() s.copy() s.extend(t) s += t s *= t s.insert(i, x) s.pop([i]) s.remove(x) s.reverse() Marko From orgnut at yahoo.com Sun Jun 4 03:48:08 2017 From: orgnut at yahoo.com (Larry Hudson) Date: Sun, 4 Jun 2017 00:48:08 -0700 Subject: Is An Element of a Sequence an Object? In-Reply-To: References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: On 06/03/2017 09:39 PM, Chris Angelico wrote: > That's a tricky thing to pin down. Since it's possible for a sequence > to contain itself, or to contain something which contains it, Like a Tardis? [Sorry, couldn't resist...] ;-) OTOH, On-Topic... It might be worth while to point out that a 'character' is NOT a data type in Python, in the same sense as it is in other languages. -- -=- Larry -=- From rustompmody at gmail.com Sun Jun 4 03:52:19 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 4 Jun 2017 00:52:19 -0700 (PDT) Subject: Is An Element of a Sequence an Object? In-Reply-To: References: Message-ID: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> On Sunday, June 4, 2017 at 12:45:23 AM UTC+5:30, Jon Forrest wrote: > I'm learning about Python. A book I'm reading about it > says "... a string in Python is a sequence. A sequence is an ordered > collection of objects". This implies that each character in a string > is itself an object. > > This doesn't seem right to me, but since I'm just learning Python > I questioned the author about this. He gave an example the displays > the ids of string slices. These ids are all different, but I think > that's because the slicing operation creates objects. > > I'd like to suggest an explanation of what a sequence is > that doesn't use the word 'object' because an object has > a specific meaning in Python. > > Am I on the right track here? Its a good sign that you are confused If you were not (feeling) confused, it would mean you are actually more so? Following is not exactly what you are disturbed by... Still closely related >>> s="a string" >>> s[0] 'a' >>> s[0][0] 'a' >>> s[0][0][0][0][0] 'a' If you prefer jargon, elements of a string are not first class From pavol.lisy at gmail.com Sun Jun 4 05:06:03 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Sun, 4 Jun 2017 11:06:03 +0200 Subject: Transitioning from Linux to Windows In-Reply-To: References: Message-ID: If I understand well then you like to have simple quick solution without need to learn much. And that you don't need to support big traffic... Maybe cherrypy ( http://cherrypy.org/ ) is what you like to look at. Probably this is good enough on your ubuntu: sudo apt-get install python3-cherrypy3 You could simply run: python3 cherry_test.py # source code bellow and open web page http://localhost:8080/ in your browser. #### cherry_test.py import string import os import os.path import cherrypy from cherrypy.lib import static localDir = os.path.dirname(__file__) absDir = os.path.join(os.getcwd(), localDir) class StringGenerator(object): @cherrypy.expose def index(self): return """
""" @cherrypy.expose def generate(self, filename='test.txt'): with open(filename, 'w') as f: f.write('test') # generate simple output file path = os.path.join(absDir, filename) return static.serve_file(path, 'application/x-download', 'attachment', os.path.basename(path)) if __name__ == '__main__': cherrypy.quickstart(StringGenerator()) ####### Simple isn't it? But don't forget to think about security! :) On 6/3/17, chitturk at uah.edu wrote: > I am looking for suggestions, ideas. > > I have developed python (3.6.x, 2.7.x) scripts that run well as a user on an > ubuntu/16.04 system - the scripts look for files, parses the files, > assembles an output for the user. > > I first cd into a particular directory on the system (where I know the files > exist) and run the script - the final result is in my working directory. > > What I am now trying to do is ... figure out the least painful way to have > other users do the same - BUT sitting in front of a Windows desktop (7 or > 10). > > Ideally, I would like to set up the user on their Windows 7/10 system so > that they can "login" to the ubuntu system (say putty) - change working > directory (to where desired) - run the script (on the ubuntu system) - and > scp the file back to the windows desktop. > > ("porting" the ubuntu script to anaconda(3) on the windows desktop IS > possible (but it has not been as easy as I had hoped!) (django and programs > like that do seem to provide a "GUI" to have python scripts run on > ubuntu/systems - but the setup looks mysterious/complicated (to me anyway)) > > I stumbled onto "paramiko" - is that a possible answer? > > Any suggestion/ideas would be greatly appreciated! > > (I am perfectly willing to stick to 3.6.x, unless there is a clean/easy > solution using 2.7.x) > > krishnan > -- > https://mail.python.org/mailman/listinfo/python-list > From steve+python at pearwood.info Sun Jun 4 05:41:00 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 04 Jun 2017 19:41:00 +1000 Subject: Is An Element of a Sequence an Object? References: <5933103c$0$1775$e4fe514c@news.kpn.nl> Message-ID: <5933d5ad$0$1600$c3e8da3$5496439d@news.astraweb.com> On Sun, 4 Jun 2017 05:48 pm, Larry Hudson wrote: > On 06/03/2017 09:39 PM, Chris Angelico wrote: >> That's a tricky thing to pin down. Since it's possible for a sequence >> to contain itself, or to contain something which contains it, > > Like a Tardis? > > [Sorry, couldn't resist...] ;-) Exactly right! I'm very fond of the model of Python objects that says they can be in multiple places at once, including inside themselves, much like the TARDIS. Unlike the TARDIS, however, they can't travel backwards in time. > OTOH, On-Topic... It might be worth while to point out that a 'character' is > NOT a data type in Python, in the same sense as it is in other languages. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Jun 4 05:51:37 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 04 Jun 2017 19:51:37 +1000 Subject: Is An Element of a Sequence an Object? References: <5933103c$0$1775$e4fe514c@news.kpn.nl> <59338d14$0$2863$c3e8da3$76491128@news.astraweb.com> Message-ID: <5933d82b$0$1600$c3e8da3$5496439d@news.astraweb.com> On Sun, 4 Jun 2017 04:50 pm, Peter Otten wrote: > Steven D'Aprano wrote: >> In Python 3, for example: >> >> >>>>> import sys >>>>> sys.getsizeof("abcde") # actual memory consumption >> 54 >>>>> sum(sys.getsizeof(c) for c in "acbde") # theoretical >> 250 >> >> >> So we can tell the two implementations apart. > > I str were implemented as a sequence of character objects it would still > report > >>>> sys.getsizeof(tuple("abcde")) > 88 > > The tuple doesn't store objects either, it holds pointers to objects and > could easily be changed to hold the values of small integers, say. Perhaps; but calling getsizeof directly is not enough to report the full memory usage of a collection. You need something like Raymond Hettinger's recursive getsizeof: code.activestate.com/recipes/577504/ Using that in Python 3.5 I get 174 bytes, and in 3.2 I get 204 bytes. The specific values will differ according to the platform, the version, and the implementation, but the conclusion should still holds. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjol at tjol.eu Sun Jun 4 06:02:16 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 4 Jun 2017 12:02:16 +0200 Subject: Is An Element of a Sequence an Object? In-Reply-To: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> References: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> Message-ID: <5933daa8$0$1564$e4fe514c@news.kpn.nl> On 04/06/17 09:52, Rustom Mody wrote: > On Sunday, June 4, 2017 at 12:45:23 AM UTC+5:30, Jon Forrest wrote: >> I'm learning about Python. A book I'm reading about it >> says "... a string in Python is a sequence. A sequence is an ordered >> collection of objects". This implies that each character in a string >> is itself an object. >> >> This doesn't seem right to me, but since I'm just learning Python >> I questioned the author about this. He gave an example the displays >> the ids of string slices. These ids are all different, but I think >> that's because the slicing operation creates objects. >> >> I'd like to suggest an explanation of what a sequence is >> that doesn't use the word 'object' because an object has >> a specific meaning in Python. >> >> Am I on the right track here? > > Its a good sign that you are confused > If you were not (feeling) confused, it would mean you are actually more so? > Following is not exactly what you are disturbed by... Still closely related > >>>> s="a string" >>>> s[0] > 'a' >>>> s[0][0] > 'a' >>>> s[0][0][0][0][0] > 'a' Also: >>> s[0] is s[0][0][0][0][0][0][0] True >>> > > If you prefer jargon, elements of a string are not first class > From jobmattcon at gmail.com Sun Jun 4 07:22:23 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 4 Jun 2017 04:22:23 -0700 (PDT) Subject: error when import private key in python Message-ID: #step 1 from Crypto.PublicKey import RSA keypair = RSA.generate(2048) alice_privkey = keypair.exportKey('PEM', 'mysecret', pkcs=1) #alice_privkey = keypair.exportKey() alice_pubkey = keypair.publickey().exportKey() text_file = open("alice_pubkey.txt", "w") text_file.write(alice_pubkey) text_file.close() keypair = RSA.generate(2048) bob_privkey = keypair.exportKey('PEM', 'mysecret2', pkcs=1) #bob_privkey = keypair.exportKey() bob_pubkey = keypair.publickey().exportKey() text_file = open("bob_pubkey.txt", "w") text_file.write(bob_pubkey) text_file.close() text_file = open("alice_privkey.pem", "w") text_file.write(alice_privkey) text_file.close() text_file = open("bob_privkey.pem", "w") text_file.write(bob_privkey) text_file.close() #step 2 #use alice public key to encrypt pubkey = RSA.importKey(alice_pubkey) alice_masterkey = pubkey.encrypt("i am Martin", None) text_file = open("alice_masterkey.txt", "w") text_file.write(bob_pubkey) text_file.close() #step 3 quit() text_file = open("alice_masterkey.txt", "r") alice_masterkey=text_file.read().replace('\n', '') text_file.close() text_file = open("alice_privkey.pem", "r") alice_privkey=text_file.read().replace('\n', '') text_file.close() privkey = RSA.importKey(alice_privkey,passphrase="mysecret") encryption_key = privkey.decrypt(alice_masterkey) >>> privkey = RSA.importKey(alice_privkey,passphrase="mysecret") Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\Crypto\PublicKey\RSA.py", line 638, in importKey if lines[1].startswith(b('Proc-Type:4,ENCRYPTED')): IndexError: list index out of range From tjol at tjol.eu Sun Jun 4 07:28:42 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 4 Jun 2017 13:28:42 +0200 Subject: error when import private key in python In-Reply-To: References: Message-ID: <5933eeea$0$1526$e4fe514c@news.kpn.nl> On 04/06/17 13:22, Ho Yeung Lee wrote: > # [snip] > alice_privkey=text_file.read().replace('\n', '') Why are you removing newlines? Does the documentation tell you to do this? >>>> privkey = RSA.importKey(alice_privkey,passphrase="mysecret") > Traceback (most recent call last): > File "", line 1, in > File "C:\Python27\lib\site-packages\Crypto\PublicKey\RSA.py", line 638, in importKey > if lines[1].startswith(b('Proc-Type:4,ENCRYPTED')): > IndexError: list index out of range > This is just a wild guess, but it looks like the package expects there to be multiple lines in the key. -- Thomas From 4kir4.1i at gmail.com Sun Jun 4 07:39:40 2017 From: 4kir4.1i at gmail.com (Akira Li) Date: Sun, 04 Jun 2017 14:39:40 +0300 Subject: Transitioning from Linux to Windows References: Message-ID: <87a85okmmb.fsf@gmail.com> chitturk at uah.edu writes: > ... > Ideally, I would like to set up the user on their Windows 7/10 system > so that they can "login" to the ubuntu system (say putty) - change > working directory (to where desired) - run the script (on the ubuntu > system) - and scp the file back to the windows desktop. > > ("porting" the ubuntu script to anaconda(3) on the windows desktop IS > possible (but it has not been as easy as I had hoped!) (django and > programs like that do seem to provide a "GUI" to have python scripts > run on ubuntu/systems - but the setup looks mysterious/complicated (to > me anyway)) > > I stumbled onto "paramiko" - is that a possible answer? > ... You could use "fabric" http://www.fabfile.org/ to automate running shell commands (such as python scripts) via ssh. From jobmattcon at gmail.com Sun Jun 4 07:45:56 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 4 Jun 2017 04:45:56 -0700 (PDT) Subject: error when import private key in python In-Reply-To: <5933eeea$0$1526$e4fe514c@news.kpn.nl> References: <5933eeea$0$1526$e4fe514c@news.kpn.nl> Message-ID: <11ec8684-c966-4ce0-a989-e7ef633c95b0@googlegroups.com> i use wb to write pubic and private key and succeed to import private, but after decrypt, it return hex number not a clear text from Crypto.PublicKey import RSA keypair = RSA.generate(2048) alice_privkey = keypair.exportKey('PEM', 'mysecret', pkcs=1) #alice_privkey = keypair.exportKey() alice_pubkey = keypair.publickey().exportKey() text_file = open("alice_pubkey.txt", "wb") text_file.write(alice_pubkey) text_file.close() keypair = RSA.generate(2048) bob_privkey = keypair.exportKey('PEM', 'mysecret2', pkcs=1) #bob_privkey = keypair.exportKey() bob_pubkey = keypair.publickey().exportKey() text_file = open("bob_pubkey.txt", "wb") text_file.write(bob_pubkey) text_file.close() text_file = open("alice_privkey.pem", "wb") text_file.write(alice_privkey) text_file.close() text_file = open("bob_privkey.pem", "wb") text_file.write(bob_privkey) text_file.close() #step 2 #use alice public key to encrypt pubkey = RSA.importKey(alice_pubkey) alice_masterkey = pubkey.encrypt("i am Martin", None) text_file = open("alice_masterkey.txt", "w") text_file.write(bob_pubkey) text_file.close() quit() python text_file = open("alice_masterkey.txt", "r") alice_masterkey=text_file.read() text_file.close() text_file = open("alice_privkey.pem", "r") alice_privkey=text_file.read() text_file.close() text_file = open("alice_pubkey.txt", "r") alice_pubkey=text_file.read() text_file.close() from Crypto.PublicKey import RSA pubkey = RSA.importKey(alice_pubkey) privkey = RSA.importKey(alice_privkey,passphrase="mysecret") encryption_key = privkey.decrypt(alice_masterkey) encryption_key quit() python text_file = open("alice_masterkey.txt", "r") alice_masterkey=text_file.read() text_file.close() text_file = open("alice_privkey.pem", "r") alice_privkey=text_file.read() text_file.close() text_file = open("alice_pubkey.txt", "r") alice_pubkey=text_file.read() text_file.close() from Crypto.PublicKey import RSA pubkey = RSA.importKey(alice_pubkey) privkey = RSA.importKey(alice_privkey,passphrase="mysecret") encryption_key = privkey.decrypt(alice_masterkey) encryption_key >>> encryption_key 'o\x94\xaeC\xe0S\x81\x05t\xd8\\A\x10?\xd2\xe5\x8c5\xc9\x1d\x14\xc7\xfd)Cs\x8b"cg\x16y\xe2\xf2L\xf1-\x08qHt\x99\xbc\xb5\xf6_\x17c\xd2&Z\x0b\xc5t\t\xe0\x8b\x03G\x10\xce\xd6\xcd\x86\xfc!\xc9i\xa2\xab\x9d\x8a\x92\xfc7 g\xa5$\x91\x85\xa2L]I\xd6\xc6\xaez\xed\x01\x95\xee)8z\x18\xc9aag\x97\x97\xb0\\)\xec"\xe4\xbez\xd3\xa8\'k%\x12\x1d\xf9\xf0\x0e\x0c\xcb\xa8\xb1\xe7}\x90\xd3\xcfs@\xc2m\x1a^\x1b0\xa7\xdd\xcd\xea\x1f\xd5\x08\x13+y"]vu\xe3\x9e\xba\x97\x10\x90S\xea\xae1=r4Yp,\xe3\xa9\xc66H\xa7\x95[M|n\x91\x98\x9c,\xc4\xf5\x7f\x8cJ\x03\xba\x04Z0lV\xe1\xd6d\xeec@\xe1\xa0\xec\x81]\xef5\r\x12\x88\xbe/\xfc\xe01\xaacn,\x8a\xe1\x14\x8a\xf4\xd85\xd8\xabD\x137\xe7T\xc4\xc1\x84b.\xd9RZ\x0e\x03#\x1e\x8dl\xe8\xe4N^\r\xf0\x1d\x8c' On Sunday, June 4, 2017 at 7:29:07 PM UTC+8, Thomas Jollans wrote: > On 04/06/17 13:22, Ho Yeung Lee wrote: > > # [snip] > > alice_privkey=text_file.read().replace('\n', '') > > Why are you removing newlines? Does the documentation tell you to do this? > > >>>> privkey = RSA.importKey(alice_privkey,passphrase="mysecret") > > Traceback (most recent call last): > > File "", line 1, in > > File "C:\Python27\lib\site-packages\Crypto\PublicKey\RSA.py", line 638, in importKey > > if lines[1].startswith(b('Proc-Type:4,ENCRYPTED')): > > IndexError: list index out of range > > > > This is just a wild guess, but it looks like the package expects there > to be multiple lines in the key. > > > -- Thomas From jobmattcon at gmail.com Sun Jun 4 07:49:41 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 4 Jun 2017 04:49:41 -0700 (PDT) Subject: error when import private key in python In-Reply-To: <5933eeea$0$1526$e4fe514c@news.kpn.nl> References: <5933eeea$0$1526$e4fe514c@news.kpn.nl> Message-ID: <32b18bd0-cd82-4315-bd69-0f3d89b188dd@googlegroups.com> i use "wb" to write public and private key and succeed to import private key but after decrypt, it is not clear text, it is hex number from Crypto.PublicKey import RSA keypair = RSA.generate(2048) alice_privkey = keypair.exportKey('PEM', 'mysecret', pkcs=1) #alice_privkey = keypair.exportKey() alice_pubkey = keypair.publickey().exportKey() text_file = open("alice_pubkey.txt", "wb") text_file.write(alice_pubkey) text_file.close() keypair = RSA.generate(2048) bob_privkey = keypair.exportKey('PEM', 'mysecret2', pkcs=1) #bob_privkey = keypair.exportKey() bob_pubkey = keypair.publickey().exportKey() text_file = open("bob_pubkey.txt", "wb") text_file.write(bob_pubkey) text_file.close() text_file = open("alice_privkey.pem", "wb") text_file.write(alice_privkey) text_file.close() text_file = open("bob_privkey.pem", "wb") text_file.write(bob_privkey) text_file.close() #step 2 #use alice public key to encrypt pubkey = RSA.importKey(alice_pubkey) alice_masterkey = pubkey.encrypt("i am Martin", None) text_file = open("alice_masterkey.txt", "w") text_file.write(bob_pubkey) text_file.close() #step 3 quit() python text_file = open("alice_masterkey.txt", "r") alice_masterkey=text_file.read() text_file.close() text_file = open("alice_privkey.pem", "r") alice_privkey=text_file.read() text_file.close() text_file = open("alice_pubkey.txt", "r") alice_pubkey=text_file.read() text_file.close() from Crypto.PublicKey import RSA pubkey = RSA.importKey(alice_pubkey) privkey = RSA.importKey(alice_privkey,passphrase="mysecret") encryption_key = privkey.decrypt(alice_masterkey) encryption_key ....hex number On Sunday, June 4, 2017 at 7:29:07 PM UTC+8, Thomas Jollans wrote: > On 04/06/17 13:22, Ho Yeung Lee wrote: > > # [snip] > > alice_privkey=text_file.read().replace('\n', '') > > Why are you removing newlines? Does the documentation tell you to do this? > > >>>> privkey = RSA.importKey(alice_privkey,passphrase="mysecret") > > Traceback (most recent call last): > > File "", line 1, in > > File "C:\Python27\lib\site-packages\Crypto\PublicKey\RSA.py", line 638, in importKey > > if lines[1].startswith(b('Proc-Type:4,ENCRYPTED')): > > IndexError: list index out of range > > > > This is just a wild guess, but it looks like the package expects there > to be multiple lines in the key. > > > -- Thomas From 4kir4.1i at gmail.com Sun Jun 4 08:22:21 2017 From: 4kir4.1i at gmail.com (Akira Li) Date: Sun, 04 Jun 2017 15:22:21 +0300 Subject: Is An Element of a Sequence an Object? References: Message-ID: <874lvwkkn6.fsf@gmail.com> Jon Forrest writes: > I'm learning about Python. A book I'm reading about it > says "... > a string in Python is a sequence. correct. > A sequence is an ordered collection of objects". correct. https://docs.python.org/3/glossary.html#term-sequence > This implies that each character in a string > is itself an object. Everything in Python is an object. If *s* is a name that refers to a str object then *s[i]* returns i-th Unicode code point from the string as a str object of length 1: >>> s = "ab" >>> s[0] 'a' It does not matter how *s* is represented internally on a chosen Python implementation: *s[0]* may return an existing object or it may create a new one on-the-fly. Here's a perfectly valid sequence of ten squares: >>> class Squares: ... def __getitem__(self, i): ... if not (-len(self) <= i < len(self)): ... raise IndexError(i) ... if i < 0: ... i += len(self) ... return i * i ... ... def __len__(self): ... return 10 >>> s = Squares() >>> s[9] 81 All 10 squares are generated on-the-fly (though all int objects already exist due to the small int caching on CPython). From paul.james.barry at gmail.com Sun Jun 4 08:38:00 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Sun, 4 Jun 2017 13:38:00 +0100 Subject: Is An Element of a Sequence an Object? In-Reply-To: <874lvwkkn6.fsf@gmail.com> References: <874lvwkkn6.fsf@gmail.com> Message-ID: Hi folks. The book being referred to is mine, Head First Python 2d Edition as published by O'Reilly Media at the end of last year. Jon has copied me on the discussion to date, so I've been following along. Here's a small PDF of the single page from my book (page 24), so that you can all see my use of the phrase in question in context: http://paulbarry.itcarlow.ie/p24.pdf - my hope is that this will inform the discussion further. Regards. Paul. On 4 June 2017 at 13:22, Akira Li <4kir4.1i at gmail.com> wrote: > Jon Forrest writes: > > > I'm learning about Python. A book I'm reading about it > > says "... > > a string in Python is a sequence. > > correct. > > > A sequence is an ordered collection of objects". > > correct. https://docs.python.org/3/glossary.html#term-sequence > > > This implies that each character in a string > > is itself an object. > > Everything in Python is an object. If *s* is a name that refers to a str > object then *s[i]* returns i-th Unicode code point from the string as a > str object of length 1: > > >>> s = "ab" > >>> s[0] > 'a' > > It does not matter how *s* is represented internally on a chosen Python > implementation: *s[0]* may return an existing object or it may create a > new one on-the-fly. > > Here's a perfectly valid sequence of ten squares: > > >>> class Squares: > ... def __getitem__(self, i): > ... if not (-len(self) <= i < len(self)): > ... raise IndexError(i) > ... if i < 0: > ... i += len(self) > ... return i * i > ... > ... def __len__(self): > ... return 10 > >>> s = Squares() > >>> s[9] > 81 > > All 10 squares are generated on-the-fly (though all int objects already > exist due to the small int caching on CPython). > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From torriem at gmail.com Sun Jun 4 12:41:52 2017 From: torriem at gmail.com (Michael Torrie) Date: Sun, 4 Jun 2017 10:41:52 -0600 Subject: Transitioning from Linux to Windows In-Reply-To: References: Message-ID: <9b11f30e-2e9b-aa9c-f7c5-26d0b43e32f8@gmail.com> On 06/03/2017 07:44 AM, chitturk at uah.edu wrote: > I stumbled onto "paramiko" - is that a possible answer? I'm confused why you would need to ssh anywhere. Command-line programs in Python should work perfectly fine in Windows and work about the same as on Linux, if you wrote them in a portable way. I don't understand the need to complicate things with ssh, django, x2go, or any of the other suggestions here. I also don't understand why you'd need the Anaconda distro. Why won't standard Python from python.org work? If you want a little GUI frontend, make it in Tkinter. From nt_mahmood at yahoo.com Sun Jun 4 13:30:19 2017 From: nt_mahmood at yahoo.com (Mahmood Naderan) Date: Sun, 4 Jun 2017 17:30:19 +0000 (UTC) Subject: openpyxl reads cell with format References: <1263996953.1990869.1496597419604.ref@mail.yahoo.com> Message-ID: <1263996953.1990869.1496597419604@mail.yahoo.com> Hello guys... With openpyxl, it seems that when the content of a cell is something like "4-Feb", then it is read as "2016-02-04 00:00:00" that looks like a calendar conversion. How can I read the cell as text instead of such an automatic conversion? Regards, Mahmood From rosuav at gmail.com Sun Jun 4 13:48:20 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 5 Jun 2017 03:48:20 +1000 Subject: Transitioning from Linux to Windows In-Reply-To: <9b11f30e-2e9b-aa9c-f7c5-26d0b43e32f8@gmail.com> References: <9b11f30e-2e9b-aa9c-f7c5-26d0b43e32f8@gmail.com> Message-ID: On Mon, Jun 5, 2017 at 2:41 AM, Michael Torrie wrote: > I'm confused why you would need to ssh anywhere. Command-line programs > in Python should work perfectly fine in Windows and work about the same > as on Linux, if you wrote them in a portable way. I don't understand > the need to complicate things with ssh, django, x2go, or any of the > other suggestions here. I also don't understand why you'd need the > Anaconda distro. Why won't standard Python from python.org work? If I'm understanding the OP correctly, the intention is to leave the script where it is on the Linux box, but trigger it (and maybe provide some parameters) from a Windows box. That by definition means some sort of network connection, hence SSH or HTTP or some other protocol. ChrisA From chitturk at uah.edu Sun Jun 4 16:56:26 2017 From: chitturk at uah.edu (chitturk at uah.edu) Date: Sun, 4 Jun 2017 13:56:26 -0700 (PDT) Subject: Transitioning from Linux to Windows In-Reply-To: References: <9b11f30e-2e9b-aa9c-f7c5-26d0b43e32f8@gmail.com> Message-ID: <22f63070-fb9f-40f5-b1d1-70f97d8a5deb@googlegroups.com> Precisely - (as Chris wrote) - the problem is NOT python itself (and yes, it can indeed run on windows machines - though I have run into some minor issues) - it is about the "user"/"users" who are pretty clueless if there is no "GUI" - several have suggested django (am going through the tutorial), cherrypy looks "simple" - perhaps even x2go ... I was looking for the "simplest" possible solution to take a script that runs on a Linux box and figure out a way to have it run from a windows client (yes, putty/ssh can work - and files can be scp'ed back and so on) (someone also referenced "fabric" (which uses paramiko) ... And yes, I was indeed trying to avoid spending time on the "UI" beyond simple text messages - asking for user input/etc and printing messages (errors) and if the computation was successful etc etc - django does seem powerful, and perhaps I may (just may) try that or cherrypy or whatever works best (!) On Sunday, June 4, 2017 at 12:48:40 PM UTC-5, Chris Angelico wrote: > On Mon, Jun 5, 2017 at 2:41 AM, Michael Torrie wrote: > > I'm confused why you would need to ssh anywhere. Command-line programs > > in Python should work perfectly fine in Windows and work about the same > > as on Linux, if you wrote them in a portable way. I don't understand > > the need to complicate things with ssh, django, x2go, or any of the > > other suggestions here. I also don't understand why you'd need the > > Anaconda distro. Why won't standard Python from python.org work? > > If I'm understanding the OP correctly, the intention is to leave the > script where it is on the Linux box, but trigger it (and maybe provide > some parameters) from a Windows box. That by definition means some > sort of network connection, hence SSH or HTTP or some other protocol. > > ChrisA From tjol at tjol.eu Sun Jun 4 18:05:36 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 5 Jun 2017 00:05:36 +0200 Subject: Transitioning from Linux to Windows In-Reply-To: <22f63070-fb9f-40f5-b1d1-70f97d8a5deb@googlegroups.com> References: <9b11f30e-2e9b-aa9c-f7c5-26d0b43e32f8@gmail.com> <22f63070-fb9f-40f5-b1d1-70f97d8a5deb@googlegroups.com> Message-ID: <593484b6$0$1742$e4fe514c@news.kpn.nl> On 04/06/17 22:56, chitturk at uah.edu wrote: > I was looking for the "simplest" possible solution to take a script that runs on a Linux box and figure out a way to have it run from a windows client Care to comment on why getting the script to run on Windows directly is so difficult? I'm sure you can build something that does the job with Django/Flask/whatever, but it doesn't sound like it's necessarily the simplest solution. >> On Mon, Jun 5, 2017 at 2:41 AM, Michael Torrie wrote: >>> I'm confused why you would need to ssh anywhere. Command-line programs >>> in Python should work perfectly fine in Windows and work about the same >>> as on Linux, if you wrote them in a portable way. I don't understand >>> the need to complicate things with ssh, django, x2go, or any of the >>> other suggestions here. I also don't understand why you'd need the >>> Anaconda distro. Why won't standard Python from python.org work? From chitturk at uah.edu Sun Jun 4 18:35:12 2017 From: chitturk at uah.edu (chitturk at uah.edu) Date: Sun, 4 Jun 2017 15:35:12 -0700 (PDT) Subject: Transitioning from Linux to Windows In-Reply-To: <593484b6$0$1742$e4fe514c@news.kpn.nl> References: <9b11f30e-2e9b-aa9c-f7c5-26d0b43e32f8@gmail.com> <22f63070-fb9f-40f5-b1d1-70f97d8a5deb@googlegroups.com> <593484b6$0$1742$e4fe514c@news.kpn.nl> Message-ID: <3481a717-e510-4ba2-9ca2-915953a19ffa@googlegroups.com> It is not difficult (for me) - I developed the scripts to analyze files created by two different linux boxes (why two boxes is a longer story) (these are linux machines connected to gene sequencers) - the users are likely going to be fairly naive when it comes to running/using programs - and while the files can be transferred to the windows machine and then analyzed/examined, was trying to see if the windows machines can be used as simply a front end (no file transfers/etc) - but yes, it is indeed a solution that can be implemented - get the files over, run the script on windows machines (even as it makes me nervous to run scripts on windows systems - with or without anaconda - I am so used to being in the Linux environment that dealing with windows makes me nervous :( On Sunday, June 4, 2017 at 5:05:49 PM UTC-5, Thomas Jollans wrote: > On 04/06/17 22:56, I wrote: > > I was looking for the "simplest" possible solution to take a script that runs on a Linux box and figure out a way to have it run from a windows client > > Care to comment on why getting the script to run on Windows directly is > so difficult? I'm sure you can build something that does the job with > Django/Flask/whatever, but it doesn't sound like it's necessarily the > simplest solution. > > > >> On Mon, Jun 5, 2017 at 2:41 AM, Michael Torrie wrote: > >>> I'm confused why you would need to ssh anywhere. Command-line programs > >>> in Python should work perfectly fine in Windows and work about the same > >>> as on Linux, if you wrote them in a portable way. I don't understand > >>> the need to complicate things with ssh, django, x2go, or any of the > >>> other suggestions here. I also don't understand why you'd need the > >>> Anaconda distro. Why won't standard Python from python.org work? From tjol at tjol.eu Sun Jun 4 19:44:30 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 5 Jun 2017 01:44:30 +0200 Subject: Transitioning from Linux to Windows In-Reply-To: <3481a717-e510-4ba2-9ca2-915953a19ffa@googlegroups.com> References: <9b11f30e-2e9b-aa9c-f7c5-26d0b43e32f8@gmail.com> <22f63070-fb9f-40f5-b1d1-70f97d8a5deb@googlegroups.com> <593484b6$0$1742$e4fe514c@news.kpn.nl> <3481a717-e510-4ba2-9ca2-915953a19ffa@googlegroups.com> Message-ID: <59349be4$0$1799$e4fe514c@news.kpn.nl> On 05/06/17 00:35, chitturk at uah.edu wrote: > It is not difficult (for me) - I developed the scripts to analyze files created by two different linux boxes (why two boxes is a longer story) (these are linux machines connected to gene sequencers) - the users are likely going to be fairly naive when it comes to running/using programs - and while the files can be transferred to the windows machine and then analyzed/examined, was trying to see if the windows machines can be used as simply a front end (no file transfers/etc) - but yes, it is indeed a solution that can be implemented - get the files over, run the script on windows machines (even as it makes me nervous to run scripts on windows systems - with or without anaconda - I am so used to being in the Linux environment that dealing with windows makes me nervous :( I see. Well, if you're going to be building a user-friendly user interface anyway, it shouldn't make much of a difference (development work load wise) whether you use web technologies (as discussed here), Tkinter, PyQt, or something else, unless you're already familiar with one of them. (if you're still toying with the idea of hacking together something with PuTTy and the users SSHing in from Windows, then a command line script on the user's PC might actually be more user-friendly...) > > On Sunday, June 4, 2017 at 5:05:49 PM UTC-5, Thomas Jollans wrote: >> On 04/06/17 22:56, I wrote: >>> I was looking for the "simplest" possible solution to take a script that runs on a Linux box and figure out a way to have it run from a windows client >> >> Care to comment on why getting the script to run on Windows directly is >> so difficult? I'm sure you can build something that does the job with >> Django/Flask/whatever, but it doesn't sound like it's necessarily the >> simplest solution. >> >> >>>> On Mon, Jun 5, 2017 at 2:41 AM, Michael Torrie wrote: >>>>> I'm confused why you would need to ssh anywhere. Command-line programs >>>>> in Python should work perfectly fine in Windows and work about the same >>>>> as on Linux, if you wrote them in a portable way. I don't understand >>>>> the need to complicate things with ssh, django, x2go, or any of the >>>>> other suggestions here. I also don't understand why you'd need the >>>>> Anaconda distro. Why won't standard Python from python.org work? > From __peter__ at web.de Mon Jun 5 02:49:43 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 05 Jun 2017 08:49:43 +0200 Subject: Is An Element of a Sequence an Object? References: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> <5933daa8$0$1564$e4fe514c@news.kpn.nl> Message-ID: Thomas Jollans wrote: > On 04/06/17 09:52, Rustom Mody wrote: >> On Sunday, June 4, 2017 at 12:45:23 AM UTC+5:30, Jon Forrest wrote: >>> I'm learning about Python. A book I'm reading about it >>> says "... a string in Python is a sequence. A sequence is an ordered >>> collection of objects". This implies that each character in a string >>> is itself an object. >>> >>> This doesn't seem right to me, but since I'm just learning Python >>> I questioned the author about this. He gave an example the displays >>> the ids of string slices. These ids are all different, but I think >>> that's because the slicing operation creates objects. >>> >>> I'd like to suggest an explanation of what a sequence is >>> that doesn't use the word 'object' because an object has >>> a specific meaning in Python. >>> >>> Am I on the right track here? >> >> Its a good sign that you are confused >> If you were not (feeling) confused, it would mean you are actually more >> so? Following is not exactly what you are disturbed by... Still closely >> related >> >>>>> s="a string" >>>>> s[0] >> 'a' >>>>> s[0][0] >> 'a' >>>>> s[0][0][0][0][0] >> 'a' > > Also: > >>>> s[0] is s[0][0][0][0][0][0][0] > True >>>> However, this is an implementation detail: >>> def is_cached(c): ... return c[0] is c[0][0] ... >>> is_cached(chr(255)) True >>> is_cached(chr(256)) False From chris at withers.org Mon Jun 5 03:16:31 2017 From: chris at withers.org (Chris Withers) Date: Mon, 5 Jun 2017 08:16:31 +0100 Subject: testfixtures 5.0.0 released! Message-ID: <3304bf78-6906-47a7-f81f-1c701c29ad1a@withers.org> Hi All, I'm pleased to announce the release of testfixtures 5.0.0 featuring the following: * Move fromnose topytest for running tests. * Switch frommanuel tosybil for checking examples in documentation. This introduces a backwards incompatible change in that|FileParser|replaces the Manuel plugin that is no longer included. * Add a ?tick? method to|test_datetime|,|test_date|and|test_time|, to advance the returned point in time, which is particularly helpful when|delta|is set to zero. The package is on PyPI and a full list of all the links to docs, issue trackers and the like can be found here: https://github.com/Simplistix/testfixtures Any questions, please do ask on the Testing in Python list or on the Simplistix open source mailing list... cheers, Chris From smilesonisamal at gmail.com Mon Jun 5 03:24:07 2017 From: smilesonisamal at gmail.com (Pradeep Patra) Date: Mon, 5 Jun 2017 12:54:07 +0530 Subject: Backward compatible of Python 2 and Python 3 Message-ID: Hi, I want to make a program which works in both python 2.7 and python 3. Is it possible? For example python 2.7 have raw_input() to accept the input from command line whereas python 3.x method is input(). So I want to make sure the same program works in both the python versions. https://stackoverflow.com/questions/26174743/python-making-a-fast-port-scanner Regards Pradeep From jussi.piitulainen at helsinki.fi Mon Jun 5 03:38:50 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Mon, 05 Jun 2017 10:38:50 +0300 Subject: Is An Element of a Sequence an Object? References: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> <5933daa8$0$1564$e4fe514c@news.kpn.nl> Message-ID: Peter Otten writes: > Thomas Jollans wrote: > >> On 04/06/17 09:52, Rustom Mody wrote: >>> On Sunday, June 4, 2017 at 12:45:23 AM UTC+5:30, Jon Forrest wrote: >>>> I'm learning about Python. A book I'm reading about it >>>> says "... a string in Python is a sequence. A sequence is an ordered >>>> collection of objects". This implies that each character in a string >>>> is itself an object. >>>> >>>> This doesn't seem right to me, but since I'm just learning Python >>>> I questioned the author about this. He gave an example the displays >>>> the ids of string slices. These ids are all different, but I think >>>> that's because the slicing operation creates objects. >>>> >>>> I'd like to suggest an explanation of what a sequence is >>>> that doesn't use the word 'object' because an object has >>>> a specific meaning in Python. >>>> >>>> Am I on the right track here? >>> >>> Its a good sign that you are confused >>> If you were not (feeling) confused, it would mean you are actually more >>> so? Following is not exactly what you are disturbed by... Still closely >>> related >>> >>>>>> s="a string" >>>>>> s[0] >>> 'a' >>>>>> s[0][0] >>> 'a' >>>>>> s[0][0][0][0][0] >>> 'a' >> >> Also: >> >>>>> s[0] is s[0][0][0][0][0][0][0] >> True >>>>> > > However, this is an implementation detail: > >>>> def is_cached(c): > ... return c[0] is c[0][0] > ... I think this works the same, and looks more dramatic to me: ... return c[0] is c[0] >>>> is_cached(chr(255)) > True >>>> is_cached(chr(256)) > False Also same thing, as far as I can see: >>> s = "\u00ff\u0100" ; (s[0] is s[0], s[1] is s[1]) (True, False) From nt_mahmood at yahoo.com Mon Jun 5 04:24:08 2017 From: nt_mahmood at yahoo.com (Mahmood Naderan) Date: Mon, 5 Jun 2017 08:24:08 +0000 (UTC) Subject: Openpyxl cell format References: <1124436147.2458126.1496651048074.ref@mail.yahoo.com> Message-ID: <1124436147.2458126.1496651048074@mail.yahoo.com> Hello guys, With openpyxl, it seems that when the content of a cell is something like "4-Feb", then it is read as "2016-02-04 00:00:00" that looks like a calendar conversion. How can I read the cell as text instead of such an automatic conversion? Regards, Mahmood From __peter__ at web.de Mon Jun 5 04:30:01 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 05 Jun 2017 10:30:01 +0200 Subject: Is An Element of a Sequence an Object? References: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> <5933daa8$0$1564$e4fe514c@news.kpn.nl> Message-ID: Jussi Piitulainen wrote: > Peter Otten writes: >> However, this is an implementation detail: >> >>>>> def is_cached(c): >> ... return c[0] is c[0][0] >> ... > > I think this works the same, and looks more dramatic to me: > > ... return c[0] is c[0] Indeed. That's how I would have written it had I understood completely the point I was trying to make ;) From tjol at tjol.eu Mon Jun 5 05:30:43 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 5 Jun 2017 11:30:43 +0200 Subject: Is An Element of a Sequence an Object? In-Reply-To: References: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> <5933daa8$0$1564$e4fe514c@news.kpn.nl> Message-ID: <5935254a$0$1715$e4fe514c@news.kpn.nl> On 05/06/17 09:38, Jussi Piitulainen wrote: > Peter Otten writes: > >> Thomas Jollans wrote: >>> Also: >>> >>>>>> s[0] is s[0][0][0][0][0][0][0] >>> True >>>>>> >> >> However, this is an implementation detail: >> >>>>> def is_cached(c): >> ... return c[0] is c[0][0] >> ... > > I think this works the same, and looks more dramatic to me: > > ... return c[0] is c[0] I love this. What I thought was going on was that single-character strings return self on [0], as they do on full-length slices. >>> c = '?' >>> c[0] is c False >>> c[:] is c True >>> c[0:1] is c True >>> I wonder why they don't do this... > >>>>> is_cached(chr(255)) >> True >>>>> is_cached(chr(256)) >> False > > Also same thing, as far as I can see: > >>>> s = "\u00ff\u0100" ; (s[0] is s[0], s[1] is s[1]) > (True, False) > From lele at metapensiero.it Mon Jun 5 05:35:50 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Mon, 05 Jun 2017 11:35:50 +0200 Subject: Backward compatible of Python 2 and Python 3 References: Message-ID: <877f0qkc95.fsf@metapensiero.it> Pradeep Patra writes: > I want to make a program which works in both python 2.7 and python 3. Is it > possible? Definitely yes. > For example python 2.7 have raw_input() to accept the input from command > line whereas python 3.x method is input(). One popular way of doing that is by using https://pypi.python.org/pypi/six/1.10.0 ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From lele at metapensiero.it Mon Jun 5 05:41:16 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Mon, 05 Jun 2017 11:41:16 +0200 Subject: Openpyxl cell format References: <1124436147.2458126.1496651048074.ref@mail.yahoo.com> <1124436147.2458126.1496651048074@mail.yahoo.com> Message-ID: <8737bekc03.fsf@metapensiero.it> Mahmood Naderan via Python-list writes: > With openpyxl, it seems that when the content of a cell is something like > "4-Feb", then it is read as "2016-02-04 00:00:00" that looks like a calendar > conversion. > > How can I read the cell as text instead of such an automatic conversion? I'm not 100% sure, but I think that is a "feature" of the spreadsheet application you used to write the document, that assumed "4-Feb" to be actually a timestamp, instead of a plain string. It would be otherwise hard to explain why you get 2016 as the year... ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From __peter__ at web.de Mon Jun 5 05:50:57 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 05 Jun 2017 11:50:57 +0200 Subject: Is An Element of a Sequence an Object? References: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> <5933daa8$0$1564$e4fe514c@news.kpn.nl> <5935254a$0$1715$e4fe514c@news.kpn.nl> Message-ID: Thomas Jollans wrote: > What I thought was going on was that single-character strings return > self on [0], as they do on full-length slices. > >>>> c = '?' >>>> c[0] is c > False >>>> c[:] is c > True >>>> c[0:1] is c > True >>>> > > I wonder why they don't do this... Perhaps noone has found a use-case that would profit from this optimization, straight-forward as it may be... From paul.james.barry at gmail.com Mon Jun 5 06:13:54 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Mon, 5 Jun 2017 11:13:54 +0100 Subject: How to change variable from list to float In-Reply-To: <94a99949-8954-433d-a566-647aa36569ae@davincibb.net> References: <94a99949-8954-433d-a566-647aa36569ae@davincibb.net> Message-ID: The value in existattn is a single element list. The single element is a float, so just refer to it in your calculation, like so: siglevfromexist = 34.8 + existattn[0] Regards. Paul. On 3 June 2017 at 15:42, Gary Barker wrote: > I have searched for a solution to this but have not found a suitable > example. > > The attached code generates this error: Traceback (most recent call last): > File "calcsignal.py", line 7, in > siglevfromexist = 34.8 + existattn > TypeError: unsupported operand type(s) for +: 'float' and 'list' > > How do I convert the list variable (i.e. existattn) to a float? > > Operating details are: > Python 3.4.2 > Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux > > The following lines are the code in calcsignal.py: > azdegpattrev = -47.40715077970316 > azattndic = {-0.9: [-0.55], -0.5: [-0.46], 3.5: [-21.0], 1.4: [-7.48], > 5.5: [-25.0], 0.0: [0.0], 1.9: [-21.0], 13.0: [-38.0], 15.0: [-39.0], 3.6: > [-25.0], 20.0: [-39.0], -1.4: [-7.48], 90.0: [-65.0], -0.4: [-0.39], 0.5: > [-0.46], 0.1: [-0.04], 1.0: [-1.31], -90.0: [-65.0], 40.0: [-42.0], 180.0: > [-65.0], 1.5: [-10.0], -1.2: [-3.69], 0.3: [-0.28], -0.3: [-0.28], 0.2: > [-0.15], -0.1: [-0.04], 1.1: [-2.34], -180.0: [-65.0], -0.2: [-0.15], 1.2: > [-3.69], -40.0: [-42.0], 0.4: [-0.39], -5.5: [-25.0], -1.5: [-10.0], -20.0: > [-39.0], 0.9: [-0.55], -3.5: [-21.0], -1.9: [-21.0], -15.0: [-39.0], -13.0: > [-38.0], 1.3: [-5.39], -1.3: [-5.39], -3.6: [-25.0], -1.0: [-1.31], -1.1: > [-2.34]} > azlist = [-0.9, -0.5, 3.5, 1.4, 5.5, 0.0, 1.9, 13.0, 15.0, 3.6, 20.0, > -1.4, 90.0, -0.4, 0.5, 0.1, 1.0, -90.0, 40.0, 180.0, 1.5, -1.2, 0.3, -0.3, > 0.2, -0.1, 1.1, -180.0, -0.2, 1.2, -40.0, 0.4, -5.5, -1.5, -20.0, 0.9, > -3.5, -1.9, -15.0, -13.0, 1.3, -1.3, -3.6, -1.0, -1.1] > azlist = [float(i) for i in azlist] > closestaz = min(azlist, key=lambda x: abs(x - azdegpattrev)) > existattn = azattndic[closestaz] > siglevfromexist = 34.8 + existattn > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From nt_mahmood at yahoo.com Mon Jun 5 06:36:26 2017 From: nt_mahmood at yahoo.com (Mahmood Naderan) Date: Mon, 5 Jun 2017 10:36:26 +0000 (UTC) Subject: Openpyxl cell format In-Reply-To: <8737bekc03.fsf@metapensiero.it> References: <1124436147.2458126.1496651048074.ref@mail.yahoo.com> <1124436147.2458126.1496651048074@mail.yahoo.com> <8737bekc03.fsf@metapensiero.it> Message-ID: <1951467314.2543439.1496658986670@mail.yahoo.com> Maybe... But specifically in my case, the excel file is exported from a web page. I think there should be a way to read the content as a pure text. Regards, Mahmood From lele at metapensiero.it Mon Jun 5 07:16:35 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Mon, 05 Jun 2017 13:16:35 +0200 Subject: Openpyxl cell format References: <1124436147.2458126.1496651048074.ref@mail.yahoo.com> <1124436147.2458126.1496651048074@mail.yahoo.com> <8737bekc03.fsf@metapensiero.it> <1951467314.2543439.1496658986670@mail.yahoo.com> Message-ID: <87r2yyit0s.fsf@metapensiero.it> Mahmood Naderan via Python-list writes: > Maybe... But specifically in my case, the excel file is exported from a web > page. I think there should be a way to read the content as a pure text. I think that openpyxl isn't doing any heuristic translation on cell's content, but I could be wrong: I bet that the cell is marked as containing a datetime value, and that's what you are getting. ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From jon+usenet at unequivocal.eu Mon Jun 5 07:29:33 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 5 Jun 2017 11:29:33 -0000 (UTC) Subject: Openpyxl cell format References: <1124436147.2458126.1496651048074.ref@mail.yahoo.com> <1124436147.2458126.1496651048074@mail.yahoo.com> <8737bekc03.fsf@metapensiero.it> <1951467314.2543439.1496658986670@mail.yahoo.com> Message-ID: On 2017-06-05, Mahmood Naderan wrote: > Maybe... But specifically in my case, the excel file is exported > from a web page. I think there should be a way to read the content > as a pure text. I have a vague memory that Excel stores dates as integers, so if you were to read the raw data you would get a number like 1496662356, which is perhaps not what you want. From nt_mahmood at yahoo.com Mon Jun 5 10:46:18 2017 From: nt_mahmood at yahoo.com (Mahmood Naderan) Date: Mon, 5 Jun 2017 14:46:18 +0000 (UTC) Subject: openpyxl reads cell with format In-Reply-To: References: <1263996953.1990869.1496597419604.ref@mail.yahoo.com> <1263996953.1990869.1496597419604@mail.yahoo.com> Message-ID: <507201548.2742022.1496673978517@mail.yahoo.com> >if the cell is an Excel date, it IS stored as a numeric As I said, the "shape" of the cell is similar to date. The content which is "4-Feb" is not a date. It is a string which I expect from cell.value to read it as "4-Feb" and nothing else. Also, I said before that the file is downloaded from a website. That means, from a button on a web page, I chose "export as excel" to download the data. I am pretty sure that auto format feature of the excel is trying to convert it as a date. So, I am looking for a way to ignore such auto formatting. Regards, Mahmood From maria.alonso.martirena at gmail.com Mon Jun 5 11:16:55 2017 From: maria.alonso.martirena at gmail.com (Maria Alonso-Martirena) Date: Mon, 5 Jun 2017 17:16:55 +0200 Subject: Download Message-ID: Good morning, You asked me to subscribe before writing to you and i've already done so. I need your help: I?ve just downloaded Python for Windows (versi?n 3.6.1.). However, when I try to open it, it just says ?modify?, ?repair? or ?uninstall?. How can I solve this problem? Why is it not working correctly? Thank you beforehand. Kind regards, Mar?a Alonso. From irmen.NOSPAM at xs4all.nl Mon Jun 5 11:44:40 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Mon, 5 Jun 2017 17:44:40 +0200 Subject: Transitioning from Linux to Windows In-Reply-To: References: Message-ID: <59357c6a$0$815$e4fe514c@news.xs4all.nl> On 3-6-2017 15:44, chitturk at uah.edu wrote: > Ideally, I would like to set up the user on their Windows 7/10 system so that they can "login" to the ubuntu system (say putty) - change working directory (to where desired) - run the script (on the ubuntu system) - and scp the file back to the windows desktop. You can use Pyro4 to directly call the python scripts running on your ubuntu system from python code on the windows machines, thereby avoiding the hassle of having to deal with a remote shell session. It does require you to run a Pyro4 daemon on the linux box and deal with security in some other way. Irmen From pkpearson at nowhere.invalid Mon Jun 5 13:01:55 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 5 Jun 2017 17:01:55 GMT Subject: Bug or intended behavior? References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On Fri, 2 Jun 2017 10:17:05 -0700 (PDT), sean.dizazzo at gmail.com wrote: [snip] >>>> print "foo %s" % 1-2 > Traceback (most recent call last): > File "", line 1, in > TypeError: unsupported operand type(s) for -: 'str' and 'int' Others have already pointed out that you're assuming the wrong precedence: Say "foo %s" % (1-2) not ("foo %s" % 1) - 2 . Personally I prefer a less compact but more explicit alternative: "foo {}".format(1-2) -- To email me, substitute nowhere->runbox, invalid->com. From rosuav at gmail.com Mon Jun 5 13:10:05 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 6 Jun 2017 03:10:05 +1000 Subject: Bug or intended behavior? In-Reply-To: References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On Tue, Jun 6, 2017 at 3:01 AM, Peter Pearson wrote: > On Fri, 2 Jun 2017 10:17:05 -0700 (PDT), sean.dizazzo at gmail.com wrote: > [snip] >>>>> print "foo %s" % 1-2 >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: unsupported operand type(s) for -: 'str' and 'int' > > Others have already pointed out that you're assuming the > wrong precedence: > > Say > "foo %s" % (1-2) > not > ("foo %s" % 1) - 2 > . > > Personally I prefer a less compact but more explicit alternative: > > "foo {}".format(1-2) The trouble with the zen of Python is that people now use the word "explicit" to mean "code that I like". What is more explicit here? It uses a method call instead of an operator, but either way, it's completely explicit that you want to populate the placeholders in a template string. ChrisA From pkpearson at nowhere.invalid Mon Jun 5 13:15:26 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 5 Jun 2017 17:15:26 GMT Subject: How to change variable from list to float References: <94a99949-8954-433d-a566-647aa36569ae@davincibb.net> Message-ID: On Mon, 5 Jun 2017 11:13:54 +0100, Paul Barry wrote: > On 3 June 2017 at 15:42, Gary Barker wrote: > >> I have searched for a solution to this but have not found a suitable >> example. >> >> The attached code generates this error: Traceback (most recent call last): >> File "calcsignal.py", line 7, in >> siglevfromexist = 34.8 + existattn >> TypeError: unsupported operand type(s) for +: 'float' and 'list' >> >> How do I convert the list variable (i.e. existattn) to a float? >> >> Operating details are: >> Python 3.4.2 >> Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux >> >> The following lines are the code in calcsignal.py: >> azdegpattrev = -47.40715077970316 >> azattndic = {-0.9: [-0.55], -0.5: [-0.46], 3.5: [-21.0], 1.4: [-7.48], >> 5.5: [-25.0], 0.0: [0.0], 1.9: [-21.0], 13.0: [-38.0], 15.0: [-39.0], 3.6: >> [-25.0], 20.0: [-39.0], -1.4: [-7.48], 90.0: [-65.0], -0.4: [-0.39], 0.5: >> [-0.46], 0.1: [-0.04], 1.0: [-1.31], -90.0: [-65.0], 40.0: [-42.0], 180.0: >> [-65.0], 1.5: [-10.0], -1.2: [-3.69], 0.3: [-0.28], -0.3: [-0.28], 0.2: >> [-0.15], -0.1: [-0.04], 1.1: [-2.34], -180.0: [-65.0], -0.2: [-0.15], 1.2: >> [-3.69], -40.0: [-42.0], 0.4: [-0.39], -5.5: [-25.0], -1.5: [-10.0], -20.0: >> [-39.0], 0.9: [-0.55], -3.5: [-21.0], -1.9: [-21.0], -15.0: [-39.0], -13.0: >> [-38.0], 1.3: [-5.39], -1.3: [-5.39], -3.6: [-25.0], -1.0: [-1.31], -1.1: >> [-2.34]} >> azlist = [-0.9, -0.5, 3.5, 1.4, 5.5, 0.0, 1.9, 13.0, 15.0, 3.6, 20.0, >> -1.4, 90.0, -0.4, 0.5, 0.1, 1.0, -90.0, 40.0, 180.0, 1.5, -1.2, 0.3, -0.3, >> 0.2, -0.1, 1.1, -180.0, -0.2, 1.2, -40.0, 0.4, -5.5, -1.5, -20.0, 0.9, >> -3.5, -1.9, -15.0, -13.0, 1.3, -1.3, -3.6, -1.0, -1.1] >> azlist = [float(i) for i in azlist] >> closestaz = min(azlist, key=lambda x: abs(x - azdegpattrev)) >> existattn = azattndic[closestaz] >> siglevfromexist = 34.8 + existattn > > The value in existattn is a single element list. The single element is a > float, so just refer to it in your calculation, like so: > > siglevfromexist = 34.8 + existattn[0] But as you do so, remember to wonder why the author of this code made the elements of azattndic *lists* of values instead of simple single values. Then, worry that someday your corrected code, which uses only the first value of the list, will be handed a list with several values. (True, it will execute without exception; but right now, today, you know that if the list length is not 1, you're confused.) -- To email me, substitute nowhere->runbox, invalid->com. From marko at pacujo.net Mon Jun 5 15:26:31 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 05 Jun 2017 22:26:31 +0300 Subject: Bug or intended behavior? References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: <874lvutevs.fsf@elektro.pacujo.net> Peter Pearson : > On Fri, 2 Jun 2017 10:17:05 -0700 (PDT), sean.dizazzo at gmail.com wrote: > [snip] >>>>> print "foo %s" % 1-2 >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: unsupported operand type(s) for -: 'str' and 'int' > > Others have already pointed out that you're assuming the > wrong precedence: > > Say > "foo %s" % (1-2) > not > ("foo %s" % 1) - 2 Python's killer feature is the dependence on indentation, that is, visual grouping. If they look like they belong together, they belong together. Interestingly, however, Python hasn't extended that principle to the expression syntax. You could have: >>> 1 + 2*3 7 >>> 1+2 * 3 9 Marko From nt_mahmood at yahoo.com Mon Jun 5 15:33:12 2017 From: nt_mahmood at yahoo.com (Mahmood Naderan) Date: Mon, 5 Jun 2017 19:33:12 +0000 (UTC) Subject: openpyxl reads cell with format In-Reply-To: References: <1263996953.1990869.1496597419604.ref@mail.yahoo.com> <1263996953.1990869.1496597419604@mail.yahoo.com> <507201548.2742022.1496673978517@mail.yahoo.com> Message-ID: <1706265192.3078330.1496691192877@mail.yahoo.com> OK thank you very much. As you said, it seems that it is too late for my python script. Regards, Mahmood On Monday, June 5, 2017 10:41 PM, Dennis Lee Bieber wrote: On Mon, 5 Jun 2017 14:46:18 +0000 (UTC), Mahmood Naderan via Python-list declaimed the following: >>if the cell is an Excel date, it IS stored as a numeric > >As I said, the "shape" of the cell is similar to date. The content which is "4-Feb" is not a date. It is a string which I expect from cell.value to read it as "4-Feb" and nothing else. > >Also, I said before that the file is downloaded from a website. That means, from a button on a web page, I chose "export as excel" to download the data. I am pretty sure that auto format feature of the excel is trying to convert it as a date. > Then you'll have to modify the Excel file before the "export" to tell IT that the column is plain text BEFORE the data goes into the column. The normal behavior for Excel is: if something looks like a date (partial or full) when entered, Excel will store it as a numeric "days from epoch" and flag the cell as a "date" field. The visual representation is up to the client -- as my sample table shows, the very same input value looks different based upon how the column is defined. > >So, I am looking for a way to ignore such auto formatting. > By the time Python sees it, it is too late -- all you have is an integer number tagged as a "date", and an import process that renders that number as a Python datetime object (which you can then render however you want https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior ) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ -- https://mail.python.org/mailman/listinfo/python-list From maria.alonso.martirena at gmail.com Mon Jun 5 15:33:46 2017 From: maria.alonso.martirena at gmail.com (Maria Alonso-Martirena) Date: Mon, 5 Jun 2017 21:33:46 +0200 Subject: Fwd: Welcome to the "Python-list" mailing list In-Reply-To: References: Message-ID: Good afternoon, I sent you an email this afternoon asking for help. As you can see in this email, I have been admited already. Could you help me with the matter I asked you please? Thanks. Kind regards, Mar?a Alonso-Martirena. ---------- Forwarded message ---------- From: Date: 2017-06-05 21:32 GMT+02:00 Subject: Welcome to the "Python-list" mailing list To: maria.alonso.martirena at gmail.com Welcome to the Python-list at python.org mailing list! The purpose of this mailing list is to support general discussions about the Python programming language. Please remember that this list is mirrored to the Usenet newsgroup comp.lang.python. For more information on the Python programming language see To post to this list, send your message to: python-list at python.org General information about the mailing list is at: https://mail.python.org/mailman/listinfo/python-list If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: https://mail.python.org/mailman/options/python-list/maria. alonso.martirena%40gmail.com You can also make such adjustments via email by sending a message to: Python-list-request at python.org with the word `help' in the subject or body (don't include the quotes), and you will get back a message with instructions. You must know your password to change your options (including changing the password, itself) or to unsubscribe without confirmation. It is: maiebsam Normally, Mailman will remind you of your python.org mailing list passwords once every month, although you can disable this if you prefer. This reminder will also include instructions on how to unsubscribe or change your account options. There is also a button on your options page that will email your current password to you. From rosuav at gmail.com Mon Jun 5 15:45:29 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 6 Jun 2017 05:45:29 +1000 Subject: Bug or intended behavior? In-Reply-To: <874lvutevs.fsf@elektro.pacujo.net> References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <874lvutevs.fsf@elektro.pacujo.net> Message-ID: On Tue, Jun 6, 2017 at 5:26 AM, Marko Rauhamaa wrote: > Peter Pearson : > >> On Fri, 2 Jun 2017 10:17:05 -0700 (PDT), sean.dizazzo at gmail.com wrote: >> [snip] >>>>>> print "foo %s" % 1-2 >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: unsupported operand type(s) for -: 'str' and 'int' >> >> Others have already pointed out that you're assuming the >> wrong precedence: >> >> Say >> "foo %s" % (1-2) >> not >> ("foo %s" % 1) - 2 > > Python's killer feature is the dependence on indentation, that is, > visual grouping. If they look like they belong together, they belong > together. > > Interestingly, however, Python hasn't extended that principle to the > expression syntax. You could have: > > >>> 1 + 2*3 > 7 > >>> 1+2 * 3 > 9 That's all very well (and cute) for situations with just two operators. What if you had four, and wanted to use whitespace to represent precedence (or the lack of it to represent parentheses, if you prefer)? Do you need to over-space the line? ChrisA From torriem at gmail.com Mon Jun 5 15:45:58 2017 From: torriem at gmail.com (Michael Torrie) Date: Mon, 5 Jun 2017 13:45:58 -0600 Subject: Bug or intended behavior? In-Reply-To: <874lvutevs.fsf@elektro.pacujo.net> References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <874lvutevs.fsf@elektro.pacujo.net> Message-ID: <977ae696-a498-f1f0-71b3-7eb3b9a1d236@gmail.com> On 06/05/2017 01:26 PM, Marko Rauhamaa wrote: > Interestingly, however, Python hasn't extended that principle to the > expression syntax. You could have: > > >>> 1 + 2*3 > 7 > >>> 1+2 * 3 > 9 And thankfully they didn't. Because it wouldn't make sense to do so. Having whitespace indenting mimics how one thinks of blocks of code and how one might wright it in pseudocode. For expressions, however, no one thinks or writes pseudo-equations without considering the order of operations. Completely different things. Maybe if the use of parenthesis was relatively recent, you might make an argument, but unlike braces, the use of parenthesis to override order of operations goes back hundreds of years, far longer than programming languages have been defining blocks. From marko at pacujo.net Mon Jun 5 16:05:41 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 05 Jun 2017 23:05:41 +0300 Subject: Bug or intended behavior? References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <874lvutevs.fsf@elektro.pacujo.net> <977ae696-a498-f1f0-71b3-7eb3b9a1d236@gmail.com> Message-ID: <87r2yyryi2.fsf@elektro.pacujo.net> Michael Torrie : > On 06/05/2017 01:26 PM, Marko Rauhamaa wrote: >> Interestingly, however, Python hasn't extended that principle to the >> expression syntax. You could have: >> >> >>> 1 + 2*3 >> 7 >> >>> 1+2 * 3 >> 9 > > And thankfully they didn't. Because it wouldn't make sense to do so. > > Having whitespace indenting mimics how one thinks of blocks of code > and how one might wright it in pseudocode. For expressions, however, > no one thinks or writes pseudo-equations without considering the order > of operations. No-one? That's what tripped up the original poster. > Completely different things. Maybe if the use of parenthesis was > relatively recent, you might make an argument, but unlike braces, the > use of parenthesis to override order of operations goes back hundreds > of years, far longer than programming languages have been defining > blocks. There are numerous traditions: Just think of the several common ways of performing calculations with electronic calculators. Marko From torriem at gmail.com Mon Jun 5 16:19:34 2017 From: torriem at gmail.com (Michael Torrie) Date: Mon, 5 Jun 2017 14:19:34 -0600 Subject: Download In-Reply-To: References: Message-ID: <0d962904-2d93-fbb4-a338-7d2bf91c5acf@gmail.com> On 06/05/2017 09:16 AM, Maria Alonso-Martirena wrote: > Good morning, > > You asked me to subscribe before writing to you and i've already done so. I > need your help: I?ve just downloaded Python for Windows (versi?n 3.6.1.). > However, when I try to open it, it just says ?modify?, ?repair? or > ?uninstall?. > > How can I solve this problem? Why is it not working correctly? You appear to be running the installer again. The installer program only needs to be run once. After that, delete it. It appears that Python is already installed and ready to go. It's probably installed to c:\python36. To use it you need to write programs in an editor of some kind and save them with a .py extensions. Then you can run them by double clicking your py programs. While you can use Windows' built-in notepad.exe, it's probably better to use a programmer's editor. For example, Notepad++ (google for it). Or use IDLE which should come with Python, and should be in your start menu. From torriem at gmail.com Mon Jun 5 16:27:30 2017 From: torriem at gmail.com (Michael Torrie) Date: Mon, 5 Jun 2017 14:27:30 -0600 Subject: Bug or intended behavior? In-Reply-To: <87r2yyryi2.fsf@elektro.pacujo.net> References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <874lvutevs.fsf@elektro.pacujo.net> <977ae696-a498-f1f0-71b3-7eb3b9a1d236@gmail.com> <87r2yyryi2.fsf@elektro.pacujo.net> Message-ID: On 06/05/2017 02:05 PM, Marko Rauhamaa wrote: > Michael Torrie : >> On 06/05/2017 01:26 PM, Marko Rauhamaa wrote: >>> Interestingly, however, Python hasn't extended that principle to the >>> expression syntax. You could have: >>> >>> >>> 1 + 2*3 >>> 7 >>> >>> 1+2 * 3 >>> 9 >> >> And thankfully they didn't. Because it wouldn't make sense to do so. >> >> Having whitespace indenting mimics how one thinks of blocks of code >> and how one might wright it in pseudocode. For expressions, however, >> no one thinks or writes pseudo-equations without considering the order >> of operations. > > No-one? That's what tripped up the original poster. I'm referring to your contrived example with only addition and multiplication and basic order of operations (multiplication and division before addition and subtraction). Everyone knows what parenthesis mean. No one uses spacing in a manner like you suggest, even for writing formulas on napkins. That's my point. Comparing Python's whitespace formatting to a theoretical replacing of parenthesis with white spacing is non sequitor, given the hundreds of years of order of operation. I don't see how anyone could seriously claim otherwise. The precedence of operators beyond the simple basics is not so set in stone of course, and varies even between programming languages. I see no easy way around this other than to encourage new programmers to a) read the basic documentation on expressions, and b) understand how to interpret tracebacks and how to break things down and try them in the REPL. From skip.montanaro at gmail.com Mon Jun 5 16:35:17 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 5 Jun 2017 15:35:17 -0500 Subject: Bug or intended behavior? In-Reply-To: <874lvutevs.fsf@elektro.pacujo.net> References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <874lvutevs.fsf@elektro.pacujo.net> Message-ID: On Mon, Jun 5, 2017 at 2:26 PM, Marko Rauhamaa wrote: > Interestingly, however, Python hasn't extended that principle to the > expression syntax. You could have: > > >>> 1 + 2*3 > 7 > >>> 1+2 * 3 > 9 In a later post, you referenced a Wikipedia page on order of operations , in which it states: *The order of operations used throughout mathematics, science, technology and many computer programming languages is expressed here...* Python's syntax is indentation-based because Guido and the people he worked with on the ABC language (I think) at CWI observed that it was easier for new programmers to understand the program's logic if it was forced to behave the way it looks ("Python, programming the way Guido indented"). So-called high-level programming languages having been available for only about 30-40 years at that point, there was still valuable research to be done on the "best" way to structure your code, especially for people new to programming. OTOH, changing the way order of operations is specified from the way settled on in mathematics over the past few hundred years (and which every algebra student learns) hardly makes sense. In this particular case, the OP simply made a mistake, one, which when explained, probably made perfect sense to him. Like many other computer languages, Python has overloaded arithmetic operators a bit, and even though '%' isn't the most common of the basic arithmetic operators, it still has the same precedence it would have if it occurred in a purely arithmetic expression: 1 + 372 % 19 Skip From tjol at tjol.eu Mon Jun 5 16:39:12 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 5 Jun 2017 22:39:12 +0200 Subject: openpyxl reads cell with format In-Reply-To: References: <1263996953.1990869.1496597419604.ref@mail.yahoo.com> <1263996953.1990869.1496597419604@mail.yahoo.com> <507201548.2742022.1496673978517@mail.yahoo.com> Message-ID: <5935c1f8$0$1786$e4fe514c@news.kpn.nl> On 05/06/17 16:46, Mahmood Naderan wrote: >> if the cell is an Excel date, it IS stored as a numeric > > As I said, the "shape" of the cell is similar to date. The content which is "4-Feb" is not a date. It is a string which I expect from cell.value to read it as "4-Feb" and nothing else. > > Also, I said before that the file is downloaded from a website. That means, from a button on a web page, I chose "export as excel" to download the data. I am pretty sure that auto format feature of the excel is trying to convert it as a date. > > > So, I am looking for a way to ignore such auto formatting. Actually, you're looking for a way to find out how Excel formats the date cell (this is presumably stored in the xlsx file) and to format the value in the same way. The content of the cell is a (numeric) date value. The cell has an associated date format. You can verify this if you open the file in Excel. If you don't believe what Excel tells you, you can have a look at the actual XML. (I have no idea how to get the date format with openpyxl, but I'm sure that's what you want to do if you need the string "4-Feb" - unless you can show us that that string is stored in the XML file) -- Thomas From suabiut at gmail.com Mon Jun 5 18:11:09 2017 From: suabiut at gmail.com (sum abiut) Date: Tue, 6 Jun 2017 09:11:09 +1100 Subject: converting Julian date to normal date Message-ID: i am using python,and django as my web framework. I use sqlalchemy to connect to MSSQL data that is running Epicor database. Epicor is using julian * date. How to you convert julian date to normal date* *cheers,* From larry.martell at gmail.com Mon Jun 5 18:14:54 2017 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 5 Jun 2017 18:14:54 -0400 Subject: converting Julian date to normal date In-Reply-To: References: Message-ID: On Mon, Jun 5, 2017 at 6:11 PM, sum abiut wrote: > i am using python,and django as my web framework. I use sqlalchemy to > connect to MSSQL data that is running Epicor database. Epicor is using > julian > > * date. How to you convert julian date to normal date* You cross posted this to the django list and I answered it there. From greg.ewing at canterbury.ac.nz Mon Jun 5 19:48:54 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 06 Jun 2017 11:48:54 +1200 Subject: Openpyxl cell format In-Reply-To: References: <1124436147.2458126.1496651048074.ref@mail.yahoo.com> <1124436147.2458126.1496651048074@mail.yahoo.com> <8737bekc03.fsf@metapensiero.it> <1951467314.2543439.1496658986670@mail.yahoo.com> Message-ID: Mahmood Naderan wrote: > Maybe... But specifically in my case, the excel file is exported from a web > page. I think there should be a way to read the content as a pure text. What form are you getting the file in? Are you being given an Excel file, or are you loading a text file into Excel? The problem is that when it reads a csv file, Excel aggressively turns anything it thinks looks like a number into a number, and anything it thinks looks like a date into a date. There are ways to avoid that, but they all rely on having access to the csv file before loading it into Excel. Once it's in Excel, the data has already been corrupted and there's nothing you can do about it. If you have access to the original data as a csv file or some other text format, I would suggest eliminating Excel altogether and just read the original file with Python. -- Greg From greg.ewing at canterbury.ac.nz Mon Jun 5 19:55:12 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 06 Jun 2017 11:55:12 +1200 Subject: openpyxl reads cell with format In-Reply-To: References: <1263996953.1990869.1496597419604.ref@mail.yahoo.com> <1263996953.1990869.1496597419604@mail.yahoo.com> <507201548.2742022.1496673978517@mail.yahoo.com> Message-ID: Mahmood Naderan wrote: > from a button on a web page, I chose "export as excel" to download the data. Do you get an option to export in any other format? CSV would be best, since you can trivially read that with Python's csv module. If Excel is the only format available, you should complain to the website maintainers that their export function is broken and is corrupting data. -- Greg From christopher_reimer at icloud.com Mon Jun 5 21:09:59 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Mon, 05 Jun 2017 18:09:59 -0700 Subject: openpyxl reads cell with format In-Reply-To: References: <1263996953.1990869.1496597419604.ref@mail.yahoo.com> <1263996953.1990869.1496597419604@mail.yahoo.com> <507201548.2742022.1496673978517@mail.yahoo.com> Message-ID: On 6/5/2017 4:55 PM, Gregory Ewing wrote: > Mahmood Naderan wrote: >> from a button on a web page, I chose "export as excel" to download >> the data. > > Do you get an option to export in any other format? > CSV would be best, since you can trivially read that > with Python's csv module. > > If Excel is the only format available, you should > complain to the website maintainers that their > export function is broken and is corrupting data. I had the opposite problem of converting the timestamp from a scraped web page into a timezone-aware timestamp to save into a CSV file that was sortable in Excel. It took me a while to figure out that my source timestamp was set to US/Eastern even though I was viewing it as US/Pacific on the website. I wrote down my thought process in a blog post that may help this situation. https://www.kickingthebitbucket.com/2017/04/04/the-python-time-zone-rabbit-hole/ Thank you, Chris Reimer From python at deborahswanson.net Tue Jun 6 00:49:52 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 5 Jun 2017 21:49:52 -0700 Subject: Namedtuple problem #32.11.d Message-ID: <00cc01d2de80$56260c80$2860e747@sambora> I have a list of namedtuples: [{Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='') . . . {Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='')] In the first section of code, I process some of the first 10 columns (r0=v0, r1=v1,...,r10=v10), and place the results in blank columns, also in the first 10 columns. In the second section of code, I'd like to put calculated values in columns 11-93 (r11='',...r63=''), in which 3 columns will be calculated 28 times for each Record. Is there a way to do this as a step in the loop: for idx, r in enumerate(records): . . . (ideally I'd like to have a loop here, or a function, that would calculate the 28 triplets and replace their empty namedtuples with their calculated values, but this is what I've failed to successfully do.) I've tried writing a function, to either calculate and write 1) all 28 triplets, 2) one triplet, or 3) one column at a time, that would be called in the body of the main loop. The general form of this function has been: def savedata(Records, row, column, data): r = Records[row] col = getattr(r, column) r = r._replace(col = data) but in every case I've tried this, it has failed. Because, I can't say r = r._replace(getattr(r, column) = data) because that gets "can't assign to a function call". And while col ( = getattr(r, column) ) holds the correct value before r = r._replace(col = data) all I get for r.column after the ._replace is an empty string, and I have no idea how that happens. I'm about to rewrite to process first 10 columns as namedtuples, and then unroll the namedtuples into a list of lists, and process the 28 triplets in two for loops. That will work, but I'm wondering if anyone can suggest a way to make my function, or one that accomplishes the same task, so as to preserve the namedtuples, and avoid the need for a second, dual nested loop to finish the second part of the problem. (Note: I have the same dilemma using recordclass instead of namedtuples.) Deborah From jobmattcon at gmail.com Tue Jun 6 02:29:40 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Mon, 5 Jun 2017 23:29:40 -0700 (PDT) Subject: how to decrypt encrypted text to a clear text Message-ID: <7df3c534-587c-4320-b65e-55a0e9086da6@googlegroups.com> i use wb to write pubic and private key and succeed to import private, but after decrypt, it return hex number not a clear text is there any more key needed? or do wb influence the result? from Crypto.PublicKey import RSA keypair = RSA.generate(2048) alice_privkey = keypair.exportKey('PEM', 'mysecret', pkcs=1) #alice_privkey = keypair.exportKey() alice_pubkey = keypair.publickey().exportKey() text_file = open("alice_pubkey.txt", "wb") text_file.write(alice_pubkey) text_file.close() keypair = RSA.generate(2048) bob_privkey = keypair.exportKey('PEM', 'mysecret2', pkcs=1) #bob_privkey = keypair.exportKey() bob_pubkey = keypair.publickey().exportKey() text_file = open("bob_pubkey.txt", "wb") text_file.write(bob_pubkey) text_file.close() text_file = open("alice_privkey.pem", "wb") text_file.write(alice_privkey) text_file.close() text_file = open("bob_privkey.pem", "wb") text_file.write(bob_privkey) text_file.close() #step 2 #use alice public key to encrypt pubkey = RSA.importKey(alice_pubkey) alice_masterkey = pubkey.encrypt("i am Martin", None) text_file = open("alice_masterkey.txt", "w") text_file.write(bob_pubkey) text_file.close() quit() python text_file = open("alice_masterkey.txt", "r") alice_masterkey=text_file.read() text_file.close() text_file = open("alice_privkey.pem", "r") alice_privkey=text_file.read() text_file.close() text_file = open("alice_pubkey.txt", "r") alice_pubkey=text_file.read() text_file.close() from Crypto.PublicKey import RSA pubkey = RSA.importKey(alice_pubkey) privkey = RSA.importKey(alice_privkey,passphrase="mysecret") encryption_key = privkey.decrypt(alice_masterkey) encryption_key quit() python text_file = open("alice_masterkey.txt", "r") alice_masterkey=text_file.read() text_file.close() text_file = open("alice_privkey.pem", "r") alice_privkey=text_file.read() text_file.close() text_file = open("alice_pubkey.txt", "r") alice_pubkey=text_file.read() text_file.close() from Crypto.PublicKey import RSA pubkey = RSA.importKey(alice_pubkey) privkey = RSA.importKey(alice_privkey,passphrase="mysecret") encryption_key = privkey.decrypt(alice_masterkey) encryption_key >>> encryption_key 'o\x94\xaeC\xe0S\x81\x05t\xd8\\A\x10?\xd2\xe5\x8c5\xc9\x1d\x14\xc7\xfd)Cs\x8b"cg\x16y\xe2\xf2L\xf1-\x08qHt\x99\xbc\xb5\xf6_\x17c\xd2&Z\x0b\xc5t\t\xe0\x8b\x03G\x10\xce\xd6\xcd\x86\xfc!\xc9i\xa2\xab\x9d\x8a\x92\xfc7 g\xa5$\x91\x85\xa2L]I\xd6\xc6\xaez\xed\x01\x95\xee)8z\x18\xc9aag\x97\x97\xb0\\)\xec"\xe4\xbez\xd3\xa8\'k%\x12\x1d\xf9\xf0\x0e\x0c\xcb\xa8\xb1\xe7}\x90\xd3\xcfs@\xc2m\x1a^\x1b0\xa7\xdd\xcd\xea\x1f\xd5\x08\x13+y"]vu\xe3\x9e\xba\x97\x10\x90S\xea\xae1=r4Yp,\xe3\xa9\xc66H\xa7\x95[M|n\x91\x98\x9c,\xc4\xf5\x7f\x8cJ\x03\xba\x04Z0lV\xe1\xd6d\xeec@\xe1\xa0\xec\x81]\xef5\r\x12\x88\xbe/\xfc\xe01\xaacn,\x8a\xe1\x14\x8a\xf4\xd85\xd8\xabD\x137\xe7T\xc4\xc1\x84b.\xd9RZ\x0e\x03#\x1e\x8dl\xe8\xe4N^\r\xf0\x1d\x8c' From __peter__ at web.de Tue Jun 6 04:30:32 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 06 Jun 2017 10:30:32 +0200 Subject: Namedtuple problem #32.11.d References: <00cc01d2de80$56260c80$2860e747@sambora> Message-ID: Deborah Swanson wrote: > [{Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='') Lovely column names ;) > Because, I can't say > > r = r._replace(getattr(r, column) = data) When r is mutable, i. e. *not* a namedtuple, you can write setattr(r, column, data) This assumes column is the column name (a string) -- and it will overwrite the current value of the attribute. If you need the current value as part of a calculation you can access it with getattr(). E. g. if you want to multiply columns r.foo and r.bar by a value stored in data: def update_record(record, columnname, data): newval = getattr(r, columnname) * data setattr(r, columnname, newval) columns = ["foo", "bar"] data = 42 for record in records: for column in columns: update_record(record, column, data) For immutable (namedtuple) rows this has to be changed: def updated_record(record, columname, data): newval = getattr(r, columnname) * data return r._update(**{columnname: newval}) for index, record in enumerate(records): for column in columns: record = updated_record(record, column, data) records[index] = record There are various approaches to make this clearer. As I have a vage memory of recommending `dict`s before I won't do it again... From anthra.norell at bluewin.ch Tue Jun 6 05:52:54 2017 From: anthra.norell at bluewin.ch (Friedrich Rentsch) Date: Tue, 6 Jun 2017 11:52:54 +0200 Subject: Finding the name of an object's source file Message-ID: Hi all, Developing a project, I have portions that work and should be assembled into a final program. Some parts don't interconnect when they should, because of my lack of rigor in managing versions. So in order to get on, I should next tidy up the mess and the way I can think of to do it is to read out the source file names of all of a working component's elements, then delete unused files and consolidate redundancy. So, the task would be to find source file names. inspect.getsource () knows which file to take the source from, but as far as I can tell, none of its methods reveals that name, if called on a command line (>>> inspect.(get source file name) ()). Object.__module__ works. Module.__file__ works, but Object.__module__.__file__ doesn't, because Object.__module__ is only a string. After one hour of googling, I believe inspect() is used mainly at runtime (introspection?) for tracing purposes. An alternative to inspect() has not come up. I guess I could grep inspect.(getsource ()), but that doesn't feel right. There's probably a simpler way. Any suggestions? Frederic From __peter__ at web.de Tue Jun 6 06:20:57 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 06 Jun 2017 12:20:57 +0200 Subject: Finding the name of an object's source file References: Message-ID: Friedrich Rentsch wrote: > Hi all, > > Developing a project, I have portions that work and should be assembled > into a final program. Some parts don't interconnect when they should, > because of my lack of rigor in managing versions. So in order to get on, > I should next tidy up the mess and the way I can think of to do it is to > read out the source file names of all of a working component's elements, > then delete unused files and consolidate redundancy. > > So, the task would be to find source file names. inspect.getsource () > knows which file to take the source from, but as far as I can tell, none > of its methods reveals that name, if called on a command line (>>> > inspect.(get source file name) ()). Object.__module__ works. > Module.__file__ works, but Object.__module__.__file__ doesn't, because > Object.__module__ is only a string. > > After one hour of googling, I believe inspect() is used mainly at > runtime (introspection?) for tracing purposes. An alternative to > inspect() has not come up. I guess I could grep inspect.(getsource ()), > but that doesn't feel right. There's probably a simpler way. Any > suggestions? What you have is a function that shows a function's source code (inspect.getsource) and a function that as part of its implementation does what you need (also inspect.getsource, incidentally). So why not combine the two to find the interesting part? >>> import inspect >>> print(inspect.getsource(inspect.getsource)) def getsource(object): ... lines, lnum = getsourcelines(object) return ''.join(lines) Next attempt: >>> print(inspect.getsource(inspect.getsourcelines)) def getsourcelines(object): ... lines, lnum = findsource(object) if ismodule(object): return lines, 0 else: return getblock(lines[lnum:]), lnum + 1 findsource? Looks like we are getting closer. >>> print(inspect.getsource(inspect.findsource)) def findsource(object): """Return the entire source file and starting line number for an object. The argument may be a module, class, method, function, traceback, frame, or code object. The source code is returned as a list of all the lines in the file and the line number indexes a line in that list. An OSError is raised if the source code cannot be retrieved.""" file = getsourcefile(object) ... Heureka: >>> import os >>> inspect.getsourcefile(os.path.split) '/usr/lib/python3.4/posixpath.py' And so much more fun than scanning the documentation :) From neilc at norwich.edu Tue Jun 6 08:29:10 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Tue, 6 Jun 2017 12:29:10 +0000 (UTC) Subject: Namedtuple problem #32.11.d References: <00cc01d2de80$56260c80$2860e747@sambora> Message-ID: On 2017-06-06, Deborah Swanson wrote: > I have a list of namedtuples: > > [{Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='') > . . . > {Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='')] > > In the first section of code, I process some of the first 10 > columns (r0=v0, r1=v1,...,r10=v10), and place the results in > blank columns, also in the first 10 columns. I too have sometimes started with a namedtuple and then found I needed to make changes to the records. I typically abandon namedtuple at this point, after only one bad experience trying to work around my choice of container. -- Neil Cerutti From m at funkyhat.org Tue Jun 6 09:52:55 2017 From: m at funkyhat.org (Matt Wheeler) Date: Tue, 06 Jun 2017 13:52:55 +0000 Subject: Finding the name of an object's source file In-Reply-To: References: Message-ID: On Tue, 6 Jun 2017 at 11:20 Peter Otten <__peter__ at web.de> wrote: > > >>> import os > >>> inspect.getsourcefile(os.path.split) > '/usr/lib/python3.4/posixpath.py' > > And so much more fun than scanning the documentation :) > Alternatively, without using inspect, we can get around `Object.__module__` being a string by importing it as a string: >>> import importlib, os >>> importlib.import_module(os.path.split.__module__).__file__ '/Users/matt/.pyenv/versions/3.6.0/lib/python3.6/posixpath.py' -- -- Matt Wheeler http://funkyh.at From anthra.norell at bluewin.ch Tue Jun 6 10:48:49 2017 From: anthra.norell at bluewin.ch (Friedrich Rentsch) Date: Tue, 6 Jun 2017 16:48:49 +0200 Subject: Finding the name of an object's source file In-Reply-To: References: Message-ID: On 06/06/2017 03:52 PM, Matt Wheeler wrote: > On Tue, 6 Jun 2017 at 11:20 Peter Otten <__peter__ at web.de> wrote: > >>>>> import os >>>>> inspect.getsourcefile(os.path.split) >> '/usr/lib/python3.4/posixpath.py' >> >> And so much more fun than scanning the documentation :) >> > Alternatively, without using inspect, we can get around `Object.__module__` > being a string by importing it as a string: > >>>> import importlib, os >>>> importlib.import_module(os.path.split.__module__).__file__ > '/Users/matt/.pyenv/versions/3.6.0/lib/python3.6/posixpath.py' Stupendous! Thanks both of you. I tried the Peter's inspect-based method on a hierarchical assembly of objects: >>> def sources (S, indent = 0): print indent * '\t' + '%-60s%s' % (S.__class__, inspect.getsourcefile (S.__class__)) if isinstance (S, WS._Association): for s in S: sources (s, indent + 1) >>> sources (M[1][1]) /home/fr/python/util/workshops.py /home/fr/python/util/workshops.py /home/fr/python/finance/position.py positions_initializer.Positions_Initializer /home/fr/python/finance/positions_initializer.py position.Position_Activity /home/fr/python/finance/position.py position.normalizer /home/fr/python/finance/position.py position.split_adjuster /home/fr/python/finance/position.py position.Journalizer /home/fr/python/finance/position.py current_work.report_unmerger /home/fr/temp/current_work.py /home/fr/python/finance/position.py position.report_name_sender /home/fr/python/finance/position.py position.Summary_Report /home/fr/python/finance/position.py workshops.File_Writer /home/fr/python/util/workshops.py Wonderful! Awesome! Matt's solution works well too. Thanks a million Frederic From larry.martell at gmail.com Tue Jun 6 11:07:29 2017 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 6 Jun 2017 11:07:29 -0400 Subject: Python program ends with 154 snakes dead; should it continue? Message-ID: I saw this headline and my first thought was what kind of program was this??? http://www.mypalmbeachpost.com/weather/new-python-program-ends-with-154-snakes-dead-should-continue/UtI3GeaBflGHEZzlfoHYgN/ We now return you to your regularly scheduled programming. From phyllismoseley3 at gmail.com Tue Jun 6 11:16:13 2017 From: phyllismoseley3 at gmail.com (Richard Moseley) Date: Tue, 6 Jun 2017 16:16:13 +0100 Subject: Namedtuple problem #32.11.d In-Reply-To: References: <00cc01d2de80$56260c80$2860e747@sambora> Message-ID: <162531b4-9a3e-1285-a341-16d9f586e413@gmail.com> On 06/06/17 13:29, Neil Cerutti wrote: > On 2017-06-06, Deborah Swanson wrote: >> I have a list of namedtuples: >> >> [{Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='') >> . . . >> {Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='')] >> >> In the first section of code, I process some of the first 10 >> columns (r0=v0, r1=v1,...,r10=v10), and place the results in >> blank columns, also in the first 10 columns. > I too have sometimes started with a namedtuple and then found I > needed to make changes to the records. I typically abandon > namedtuple at this point, after only one bad experience trying to > work around my choice of container. > In a package that I'm working on that enables access natively to legacy C-ISAM files, I've found that I need to create a form of a namedtuple that makes use of descriptors to access each of the fields within a record using attribute lookup, but to also create the record object at runtime using a description of the fields read from another file (C-ISAM works by using offsets within a record and does not natively store a mapping between name and offset). I found that using a metaclass to store the fields in a OrderedDict enabled me to embed a namedtuple to return the record back to the rest of the package. The code can be found in pyisam/table/record.py from http://github.com/rpmoseley/pyisam.git. The code is written for 3.x but the code path for versions before 3.6 should work for 2.7. From israel at ravnalaska.net Tue Jun 6 11:36:09 2017 From: israel at ravnalaska.net (israel) Date: Tue, 06 Jun 2017 07:36:09 -0800 Subject: Psycopg2 pool clarification In-Reply-To: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> References: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> Message-ID: <4cdc7e80393a42f209e8ca0742ace3fa@ravnalaska.net> Since I've gotten no replies to this, I was wondering if someone could at least confirm which behavior (my expected or my observed) is *supposed* to be the correct? Should a psycopg2 pool keep connections open when returned to the pool (if closed is False), or should it close them as long as there is more than minconn open? i.e is my observed behavior a bug or a feature? On 2017-06-02 15:06, Israel Brewster wrote: > I've been using the psycopg2 pool class for a while now, using code > similar to the following: > >>>> pool=ThreadedConnectionPool(0,5,) >>>> conn1=pool.getconn() >>>> >>>> pool.putconn(conn1) > .... repeat later, or perhaps "simultaneously" in a different thread. > > and my understanding was that the pool logic was something like the > following: > > - create a "pool" of connections, with an initial number of > connections equal to the "minconn" argument > - When getconn is called, see if there is an available connection. If > so, return it. If not, open a new connection and return that (up to > "maxconn" total connections) > - When putconn is called, return the connection to the pool for > re-use, but do *not* close it (unless the close argument is specified > as True, documentation says default is False) > - On the next request to getconn, this connection is now available and > so no new connection will be made > - perhaps (or perhaps not), after some time, unused connections would > be closed and purged from the pool to prevent large numbers of only > used once connections from laying around. > > However, in some testing I just did, this doesn't appear to be the > case, at least based on the postgresql logs. Running the following > code: > >>>> pool=ThreadedConnectionPool(0,5,) >>>> conn1=pool.getconn() >>>> conn2=pool.getconn() >>>> pool.putconn(conn1) >>>> pool.putconn(conn2) >>>> conn3=pool.getconn() >>>> pool.putconn(conn3) > > produced the following output in the postgresql log: > > 2017-06-02 14:30:26 AKDT LOG: connection received: host=::1 port=64786 > 2017-06-02 14:30:26 AKDT LOG: connection authorized: user=logger > database=flightlogs > 2017-06-02 14:30:35 AKDT LOG: connection received: host=::1 port=64788 > 2017-06-02 14:30:35 AKDT LOG: connection authorized: user=logger > database=flightlogs > 2017-06-02 14:30:46 AKDT LOG: disconnection: session time: > 0:00:19.293 user=logger database=flightlogs host=::1 port=64786 > 2017-06-02 14:30:53 AKDT LOG: disconnection: session time: > 0:00:17.822 user=logger database=flightlogs host=::1 port=64788 > 2017-06-02 14:31:15 AKDT LOG: connection received: host=::1 port=64790 > 2017-06-02 14:31:15 AKDT LOG: connection authorized: user=logger > database=flightlogs > 2017-06-02 14:31:20 AKDT LOG: disconnection: session time: > 0:00:05.078 user=logger database=flightlogs host=::1 port=64790 > > Since I set the maxconn parameter to 5, and only used 3 connections, I > wasn't expecting to see any disconnects - and yet as soon as I do > putconn, I *do* see a disconnection. Additionally, I would have > thought that when I pulled connection 3, there would have been two > connections available, and so it wouldn't have needed to connect > again, yet it did. Even if I explicitly say close=False in the putconn > call, it still closes the connection and has to open > > What am I missing? From this testing, it looks like I get no benefit > at all from having the connection pool, unless you consider an upper > limit to the number of simultaneous connections a benefit? :-) Maybe a > little code savings from not having to manually call connect and close > after each connection, but that's easily gained by simply writing a > context manager. I could get *some* limited benefit by raising the > minconn value, but then I risk having connections that are *never* > used, yet still taking resources on the DB server. > > Ideally, it would open as many connections as are needed, and then > leave them open for future requests, perhaps with an "idle" timeout. > Is there any way to achieve this behavior? > > ----------------------------------------------- > Israel Brewster > Systems Analyst II > Ravn Alaska > 5245 Airport Industrial Rd > Fairbanks, AK 99709 > (907) 450-7293 > ----------------------------------------------- From skip.montanaro at gmail.com Tue Jun 6 12:12:36 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 6 Jun 2017 11:12:36 -0500 Subject: Python program ends with 154 snakes dead; should it continue? In-Reply-To: References: Message-ID: > I saw this headline and my first thought was what kind of program was this??? > > http://www.mypalmbeachpost.com/weather/new-python-program-ends-with-154-snakes-dead-should-continue/UtI3GeaBflGHEZzlfoHYgN/ If only getting rid of invasive species was a SMOP. Skip From rosuav at gmail.com Tue Jun 6 12:31:56 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Jun 2017 02:31:56 +1000 Subject: Python program ends with 154 snakes dead; should it continue? In-Reply-To: References: Message-ID: On Wed, Jun 7, 2017 at 1:07 AM, Larry Martell wrote: > I saw this headline and my first thought was what kind of program was this??? > > http://www.mypalmbeachpost.com/weather/new-python-program-ends-with-154-snakes-dead-should-continue/UtI3GeaBflGHEZzlfoHYgN/ > > We now return you to your regularly scheduled programming. Well, no. Once a Python program ends, it should not continue. At that point, you're supposed to go back to the OS. ChrisA From pkpearson at nowhere.invalid Tue Jun 6 12:44:59 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 6 Jun 2017 16:44:59 GMT Subject: Bug or intended behavior? References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On Tue, 6 Jun 2017 03:10:05 +1000, Chris Angelico wrote: > On Tue, Jun 6, 2017 at 3:01 AM, Peter Pearson wrote: [snip] >> Say >> "foo %s" % (1-2) >> not >> ("foo %s" % 1) - 2 >> . >> >> Personally I prefer a less compact but more explicit alternative: >> >> "foo {}".format(1-2) > > The trouble with the zen of Python is that people now use the word > "explicit" to mean "code that I like". What is more explicit here? It > uses a method call instead of an operator, but either way, it's > completely explicit that you want to populate the placeholders in a > template string. I meant simply that when you call "format", you don't have to think about the precedence of the "%" operator. Maybe "explicit" isn't the right word; I just make fewer mistakes when I stick with things that I use all the time (function calls, in this case). -- To email me, substitute nowhere->runbox, invalid->com. From stephen_tucker at sil.org Tue Jun 6 13:09:13 2017 From: stephen_tucker at sil.org (Stephen Tucker) Date: Tue, 6 Jun 2017 18:09:13 +0100 Subject: Unhelpful error message Message-ID: Hi, I have just been thrown through an unecessary loop because of an unhelpful error message. I am running Python 2.7.10 on a Windows 10 machine. I incorporate sample output from IDLE: ~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> print float ("123.456") 123.456 >>> print float ("") Traceback (most recent call last): File "", line 1, in print float ("") ValueError: could not convert string to float: >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~ The unhelpful message is the response that the system gives when you try to convert a null string to a floating point number. Now, in the example I have given here, it is obvious that the string is null, but in the example that threw me into a loop, it was not obvious. It would be more helpful is the system replied: ValueError: could not convert null string to float Any chance of that becoming the case? Stephen Tucker. Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> From tjreedy at udel.edu Tue Jun 6 13:16:00 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 6 Jun 2017 13:16:00 -0400 Subject: Bug or intended behavior? In-Reply-To: References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On 6/5/2017 1:01 PM, Peter Pearson wrote: > On Fri, 2 Jun 2017 10:17:05 -0700 (PDT), sean.dizazzo at gmail.com wrote: > [snip] >>>>> print "foo %s" % 1-2 >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: unsupported operand type(s) for -: 'str' and 'int' > > Others have already pointed out that you're assuming the > wrong precedence: > > Say > "foo %s" % (1-2) > not > ("foo %s" % 1) - 2 > . > > Personally I prefer a less compact but more explicit alternative: > > "foo {}".format(1-2) More compact: >>> f'foo {1-2}' 'foo -1' -- Terry Jan Reedy From rosuav at gmail.com Tue Jun 6 13:19:27 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Jun 2017 03:19:27 +1000 Subject: Bug or intended behavior? In-Reply-To: References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On Wed, Jun 7, 2017 at 2:44 AM, Peter Pearson wrote: > On Tue, 6 Jun 2017 03:10:05 +1000, Chris Angelico wrote: >> On Tue, Jun 6, 2017 at 3:01 AM, Peter Pearson wrote: > [snip] >>> Say >>> "foo %s" % (1-2) >>> not >>> ("foo %s" % 1) - 2 >>> . >>> >>> Personally I prefer a less compact but more explicit alternative: >>> >>> "foo {}".format(1-2) >> >> The trouble with the zen of Python is that people now use the word >> "explicit" to mean "code that I like". What is more explicit here? It >> uses a method call instead of an operator, but either way, it's >> completely explicit that you want to populate the placeholders in a >> template string. > > I meant simply that when you call "format", you don't have to think > about the precedence of the "%" operator. Maybe "explicit" isn't > the right word; I just make fewer mistakes when I stick with things > that I use all the time (function calls, in this case). Thank you, that's a more accurate way to describe it. If you don't want to memorize precedence but like the simplicity of percent formatting, try this: def fmt(str, args): return str % args fmt("foo %s", 1-2) Also avoids the problem of formatting a tuple the wrong way by mistake. ChrisA From rosuav at gmail.com Tue Jun 6 13:27:36 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Jun 2017 03:27:36 +1000 Subject: Unhelpful error message In-Reply-To: References: Message-ID: On Wed, Jun 7, 2017 at 3:09 AM, Stephen Tucker wrote: > I have just been thrown through an unecessary loop because of an unhelpful > error message. > > I am running Python 2.7.10 on a Windows 10 machine. > >>>> print float ("") > > Traceback (most recent call last): > File "", line 1, in > print float ("") > ValueError: could not convert string to float: >>>> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > The unhelpful message is the response that the system gives when you try to > convert a null string to a floating point number. Now, in the example I > have given here, it is obvious that the string is null, but in the example > that threw me into a loop, it was not obvious. > > It would be more helpful is the system replied: > > ValueError: could not convert null string to float > > Any chance of that becoming the case? > Actually it's already telling you, but the empty string isn't easy to read there. Compare: >>> float("asdf") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: asdf >>> float("asd") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: asd >>> float("as") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: as >>> float("a") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: a >>> float("") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: You could ask for the empty string to be special-cased for clarity, but the information is definitely there. Or perhaps showing the repr of the string would be clearer. ChrisA From __peter__ at web.de Tue Jun 6 13:52:00 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 06 Jun 2017 19:52 +0200 Subject: Unhelpful error message References: Message-ID: Stephen Tucker wrote: > Hi, > > I have just been thrown through an unecessary loop because of an unhelpful > error message. > > I am running Python 2.7.10 on a Windows 10 machine. > > I incorporate sample output from IDLE: > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >>>> print float ("123.456") > 123.456 >>>> print float ("") > > Traceback (most recent call last): > File "", line 1, in > print float ("") > ValueError: could not convert string to float: >>>> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~ > The unhelpful message is the response that the system gives when you try > to convert a null string to a floating point number. Now, in the example I > have given here, it is obvious that the string is null, but in the example > that threw me into a loop, it was not obvious. > > It would be more helpful is the system replied: > > ValueError: could not convert null string to float > > Any chance of that becoming the case? "null string" sounds a lot like C. The most obvious fix in Python would be to apply repr() which Python 3 already does for some strings... >>> float("-") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: '-' >>> float("1x") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: '1x' ...but not the empty string: >>> float("") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: Maybe there were some backward compatibility concerns that I lack the fantasy to imagine. From ablacktshirt at gmail.com Tue Jun 6 14:16:53 2017 From: ablacktshirt at gmail.com (Yubin Ruan) Date: Wed, 7 Jun 2017 02:16:53 +0800 Subject: Download In-Reply-To: References: Message-ID: <20170606181650.GA3117@HP> On Mon, Jun 05, 2017 at 05:16:55PM +0200, Maria Alonso-Martirena wrote: > Good morning, > > > > You asked me to subscribe before writing to you and i've already done so. I > need your help: I?ve just downloaded Python for Windows (versi?n 3.6.1.). > However, when I try to open it, it just says ?modify?, ?repair? or > ?uninstall?. > > How can I solve this problem? Why is it not working correctly? A quick search of "Python Tutorial" in Youtube will be helpful. -- Yubin From skip.montanaro at gmail.com Tue Jun 6 14:22:52 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 6 Jun 2017 13:22:52 -0500 Subject: Unhelpful error message In-Reply-To: References: Message-ID: On Tue, Jun 6, 2017 at 12:27 PM, Chris Angelico wrote: > Or perhaps showing the repr of the string would be clearer. This would definitely be better, I think, and require no special casing of the empty string. Still, I doubt this would ever be backported for fear of breaking code might be out there somewhere which relies on the current behavior. At best, it might be added to the next 3.x release (3.7?). Skip From rosuav at gmail.com Tue Jun 6 14:29:40 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Jun 2017 04:29:40 +1000 Subject: Unhelpful error message In-Reply-To: References: Message-ID: On Wed, Jun 7, 2017 at 4:22 AM, Skip Montanaro wrote: > On Tue, Jun 6, 2017 at 12:27 PM, Chris Angelico wrote: >> Or perhaps showing the repr of the string would be clearer. > > This would definitely be better, I think, and require no special > casing of the empty string. Still, I doubt this would ever be > backported for fear of breaking code might be out there somewhere > which relies on the current behavior. At best, it might be added to > the next 3.x release (3.7?). Yeah. Unfortunately this exception doesn't make good use of its args, so you can't get programmatic information from it (this is true in 3.7 too actually). So if a change were to be made, I'd recommend putting the actual string into e.args[1] or something, and then the message shouldn't need to be parsed. I'm not sure what the backward compat guarantees on exception messages are, though, so it is possible that this might be changed in 2.7. ChrisA From tomuxiong at gmx.com Tue Jun 6 14:31:02 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Tue, 6 Jun 2017 11:31:02 -0700 Subject: Unhelpful error message In-Reply-To: References: Message-ID: <12f4b2d5-0ebc-5132-d560-fc2e8ed35fbf@gmx.com> On 06/06/2017 11:22 AM, Skip Montanaro wrote: > On Tue, Jun 6, 2017 at 12:27 PM, Chris Angelico wrote: >> Or perhaps showing the repr of the string would be clearer. > > This would definitely be better, I think, and require no special > casing of the empty string. Still, I doubt this would ever be > backported for fear of breaking code might be out there somewhere > which relies on the current behavior. At best, it might be added to > the next 3.x release (3.7?). > > Skip > Does it make sense to open a bug report? I'm always a bit intimidated by contributing to python, but the following patch does seem to work (against a new clone of the master branch): ---------------------- $ git diff diff --git a/Python/pystrtod.c b/Python/pystrtod.c index 64d0c52..b3d84e4 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -355,9 +355,15 @@ PyOS_string_to_double(const char *s, "could not convert string to float: " "%.200s", s); else if (fail_pos == s) - PyErr_Format(PyExc_ValueError, - "could not convert string to float: " - "%.200s", s); + if (*s == 0) { + PyErr_Format(PyExc_ValueError, + "could not convert string to float: ''"); + } + else { + PyErr_Format(PyExc_ValueError, + "could not convert string to float: " + "%.200s", s); + } else if (errno == ERANGE && fabs(x) >= 1.0 && overflow_exception) PyErr_Format(overflow_exception, "value too large to convert to float: " ---------------------- My changes feel a bit hacky. I wanted to just drop a straight repr() in, but I didn't want to change the code too much since I assume the string formatting is already there for a reason (e.g. "%.200s"). In any case, that patch seems to be working at first glance: >>> float('') Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: '' >>> float('a') Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: 'a' >>> float('1') 1.0 Cheers, Thomas From jon+usenet at unequivocal.eu Tue Jun 6 14:38:25 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 6 Jun 2017 18:38:25 -0000 (UTC) Subject: Unhelpful error message References: Message-ID: On 2017-06-06, Peter Otten <__peter__ at web.de> wrote: > ...but not the empty string: > >>>> float("") > Traceback (most recent call last): > File "", line 1, in > ValueError: could not convert string to float: > > Maybe there were some backward compatibility concerns that I lack the > fantasy to imagine. It's a little odd, PyFloat_FromString just calls: PyErr_Format(PyExc_ValueError, "could not convert string to float: %R", v); which should be appending the repr() of the argument to the error message, and there's no code I can see anywhere that would be special-casing the empty string and avoiding inserting "''". From jon+usenet at unequivocal.eu Tue Jun 6 14:46:14 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 6 Jun 2017 18:46:14 -0000 (UTC) Subject: Unhelpful error message References: <12f4b2d5-0ebc-5132-d560-fc2e8ed35fbf@gmx.com> Message-ID: On 2017-06-06, Thomas Nyberg wrote: > My changes feel a bit hacky. I wanted to just drop a straight repr() in, > but I didn't want to change the code too much since I assume the string > formatting is already there for a reason (e.g. "%.200s"). Just change the '%.200s' to '%.200R' and it should work. The '.200' isn't necessary to avoid buffer-overflow or anything, but an error message of unlimited length is probably a good thing to avoid anyway ;-) From tomuxiong at gmx.com Tue Jun 6 14:57:26 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Tue, 6 Jun 2017 11:57:26 -0700 Subject: Unhelpful error message In-Reply-To: References: Message-ID: <477fdb5b-8884-04b2-d54b-e75504d5fe1b@gmx.com> On 06/06/2017 11:38 AM, Jon Ribbens wrote: > On 2017-06-06, Peter Otten <__peter__ at web.de> wrote: >> ...but not the empty string: >> >>>>> float("") >> Traceback (most recent call last): >> File "", line 1, in >> ValueError: could not convert string to float: >> >> Maybe there were some backward compatibility concerns that I lack the >> fantasy to imagine. > > It's a little odd, PyFloat_FromString just calls: > > PyErr_Format(PyExc_ValueError, > "could not convert string to float: %R", v); > > which should be appending the repr() of the argument to the error > message, and there's no code I can see anywhere that would be > special-casing the empty string and avoiding inserting "''". > If the string is empty, then the line: if (end != last) { does not evaluate to true, so you end up skipping the error you're referring to and instead executing these lines: else if (x == -1.0 && PyErr_Occurred()) { return NULL; } So maybe the logic should be fixed here since the empty string is basically taking a separate logical path than it philosophically should (imo). Cheers, Thomas From tomuxiong at gmx.com Tue Jun 6 15:01:21 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Tue, 6 Jun 2017 12:01:21 -0700 Subject: Unhelpful error message In-Reply-To: References: <12f4b2d5-0ebc-5132-d560-fc2e8ed35fbf@gmx.com> Message-ID: On 06/06/2017 11:46 AM, Jon Ribbens wrote: > On 2017-06-06, Thomas Nyberg wrote: >> My changes feel a bit hacky. I wanted to just drop a straight repr() in, >> but I didn't want to change the code too much since I assume the string >> formatting is already there for a reason (e.g. "%.200s"). > > Just change the '%.200s' to '%.200R' and it should work. The '.200' > isn't necessary to avoid buffer-overflow or anything, but an error > message of unlimited length is probably a good thing to avoid anyway ;-) > That causes a segfault for me. I.e. if I use this patch: --------------------- $ git diff master diff --git a/Python/pystrtod.c b/Python/pystrtod.c index 64d0c52..c62db6b 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -357,7 +357,7 @@ PyOS_string_to_double(const char *s, else if (fail_pos == s) PyErr_Format(PyExc_ValueError, "could not convert string to float: " - "%.200s", s); + "%.200R", s); else if (errno == ERANGE && fabs(x) >= 1.0 && overflow_exception) PyErr_Format(overflow_exception, "value too large to convert to float: " --------------------- Is it because the string object `s` is a const char* string and not a python string and hence the raw R isn't working right? Regardless, I think that the logic should probably be handled above this function (i.e. as I posted in my other reply to you). In any case, thanks for the help! Cheers, Thomas From jon+usenet at unequivocal.eu Tue Jun 6 15:10:53 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 6 Jun 2017 19:10:53 -0000 (UTC) Subject: Unhelpful error message References: <12f4b2d5-0ebc-5132-d560-fc2e8ed35fbf@gmx.com> Message-ID: On 2017-06-06, Thomas Nyberg wrote: > On 06/06/2017 11:46 AM, Jon Ribbens wrote: >> On 2017-06-06, Thomas Nyberg wrote: >>> My changes feel a bit hacky. I wanted to just drop a straight repr() in, >>> but I didn't want to change the code too much since I assume the string >>> formatting is already there for a reason (e.g. "%.200s"). >> >> Just change the '%.200s' to '%.200R' and it should work. The '.200' >> isn't necessary to avoid buffer-overflow or anything, but an error >> message of unlimited length is probably a good thing to avoid anyway ;-) > > That causes a segfault for me. I.e. if I use this patch: Oh yes, sorry, I didn't realise that this convert-string-to-float function in the Python source (as opposed to the other convert-string-to-float function in the Python source) was taking a char* not a PyObject*. From FredFishbin at hotmail.com Tue Jun 6 15:36:51 2017 From: FredFishbin at hotmail.com (Fred Fishbin) Date: Tue, 06 Jun 2017 14:36:51 -0500 Subject: Access flles on a Android device from Windows PC Message-ID: Hi I want to write little program that my friend can run - he'll plug a USB drive into his Windows 7 PC, plug his phone in a USB port on same PC, then run my program and it'll xfer some audiobook files over for him. I plugged the USB drive in and it became "G:\", but the phone plugged in and became just "SAMSUNG-SM-G930V", no drive designation, just a Portable Media Device. Windows Explore can go there so the files are accessible, the phone isn't looking him out, but what do I use for a path? I tried several different listdir()'s nothing worked. Thoughts? thanks, Freddie From python at deborahswanson.net Tue Jun 6 15:57:39 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 6 Jun 2017 12:57:39 -0700 Subject: Namedtuple problem #32.11.d In-Reply-To: Message-ID: <004001d2deff$27300190$2860e747@sambora> Peter Otten wrote, on Tuesday, June 06, 2017 1:31 AM > > Deborah Swanson wrote: > > > [{Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='') > > Lovely column names ;) Not very sexy names, I agree ;) The columns do have real names. The first 10 are friendly and semi-friendly names, but the names for the 28 triplets are each comprised of a root word with a prefix and a suffix, so they're quite cryptic. Copying them all out would have overwhelmed the first half of this message and obscured that the columns are divided into two groups, each group needing a different programmatic approach. So I went with simple numerically based names that permitted the use of ellipses. But I think you knew this, and were just teasing ;) > > Because, I can't say > > > > r = r._replace(getattr(r, column) = data) > > When r is mutable, i. e. *not* a namedtuple, you can write > > setattr(r, column, data) > > This assumes column is the column name (a string) -- and it will overwrite > the current value of the attribute. I'll have to try this using recordclass instead of namedtuple. After the import, it's only a one line change to transform namedtuple code to recordclass code. Change Record = namedtuple("Record", fieldnames) to Record = recordclass("Record", fieldnames) and recordclass is written so elegantly that your namedtuple code functions exactly the same as recordclass code as it did when it was namedtuple code. (Well, I've only tried it in this one case, but from reading the recordclass tutorial, I think that will hold up to be true in most if not all cases.) So you can easily switch back and forth between them. > If you need the current value as part of a calculation you can access it > with getattr(). E. g. if you want to multiply columns r.foo and r.bar by a > value stored in data: > > def update_record(record, columnname, data): > newval = getattr(r, columnname) * data > setattr(r, columnname, newval) > > columns = ["foo", "bar"] > data = 42 > > for record in records: > for column in columns: > update_record(record, column, data) Most excellent, and I'll have to also try this. I'd never seen getattr or setattr last time we attempted a namedtuple problem, and even though I understand them a little better now, I still haven't made a lot of headway in using them. This will be a good chance to see another way they can be successfully used, so thank you. > For immutable (namedtuple) rows this has to be changed: > > def updated_record(record, columname, data): > newval = getattr(r, columnname) * data > return r._update(**{columnname: newval}) > > for index, record in enumerate(records): > for column in columns: > record = updated_record(record, column, data) > records[index] = record Again, most excellent, and I'll try this too. It is one of my primary goals to learn what standard Python can do, both for it's own sake, and so I can more fully appreciate what the packages and extensions have to offer. So a possible solution using native namedtuple code is much appreciated. > There are various approaches to make this clearer. As I have a vage memory > of recommending `dict`s before I won't do it again... And I'm truly sorry your recommendation of several dict solutions last time around baffled me so profoundly. The online introductory courses in Python that I'd taken only gave us a very cursory introduction to simple uses of dicts, and I had no idea that they could be used in so many complex and mysterious ways. But I've been working with dicts, and particularly with your examples of their use. I think I have a much better understanding of most of your examples now, but not all. I also recall your mentioning that you would solve the previous namedtuple problem I asked about with dicts throughout, bypassing namedtuples altogether. I was quite intrigued with that idea, but since I lack sufficient experience with dicts I was reluctant to ask you to elaborate. I suspect you are hinting at a dicts-throughout solution here, but I can only guess what you might mean. I would be very interested in seeing what you have in mind, if you are so inclined. Though I think I would need some time to work with it and understand it better before I attempted to reply. Thank you again for your continued help with this class of problems. And I think it would be best for me to simply take in what you have to say and work with it for while, rather than trying to interact while my ideas about what may be going on are still fuzzy and I'm easily confused. Just trying to spare us both some needless frustration ;) Deborah From rosuav at gmail.com Tue Jun 6 16:06:10 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Jun 2017 06:06:10 +1000 Subject: Unhelpful error message In-Reply-To: <12f4b2d5-0ebc-5132-d560-fc2e8ed35fbf@gmx.com> References: <12f4b2d5-0ebc-5132-d560-fc2e8ed35fbf@gmx.com> Message-ID: On Wed, Jun 7, 2017 at 4:31 AM, Thomas Nyberg wrote: > Does it make sense to open a bug report? I'm always a bit intimidated by > contributing to python, but the following patch does seem to work > (against a new clone of the master branch): Your original post talked about 2.7; when you say "master branch", do you mean that you tested this against 3.x? I've opened a thread on python-ideas to talk about adding more info to the exception object. Feel free to come along and join the discussion; patches would ultimately want to be on the bug tracker (or, better still, a GitHub pull request), but you'll also want to know whether people even think this is a good idea, and the mailing list is great for that https://mail.python.org/mailman/listinfo/python-ideas ChrisA From python at deborahswanson.net Tue Jun 6 16:06:33 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 6 Jun 2017 13:06:33 -0700 Subject: Namedtuple problem #32.11.d In-Reply-To: Message-ID: <004601d2df00$656e09b0$2860e747@sambora> Neil Cerutti wrote, on Tuesday, June 06, 2017 5:29 AM > > On 2017-06-06, Deborah Swanson wrote: > > I have a list of namedtuples: > > > > [{Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='') > > . . . > > {Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='')] > > > > In the first section of code, I process some of the first > 10 columns > > (r0=v0, r1=v1,...,r10=v10), and place the results in blank columns, > > also in the first 10 columns. > > I too have sometimes started with a namedtuple and then found > I needed to make changes to the records. I typically abandon > namedtuple at this point, after only one bad experience > trying to work around my choice of container. > > -- > Neil Cerutti I can appreciate that reaction. Guess I'm a bit of a bulldog though (right ot wrong), and the concept of namedtuples is so ideally suited for the Excel spreadsheet conversions I'm working on, I'll keep on pushing the boundaries to see how they can be made to work. ;) Deborah From python at deborahswanson.net Tue Jun 6 16:27:33 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 6 Jun 2017 13:27:33 -0700 Subject: Namedtuple problem #32.11.d In-Reply-To: <162531b4-9a3e-1285-a341-16d9f586e413@gmail.com> Message-ID: <005301d2df03$54c3cde0$2860e747@sambora> Richard Moseley wrote, on Tuesday, June 06, 2017 8:16 AM > > On 06/06/17 13:29, Neil Cerutti wrote: > > On 2017-06-06, Deborah Swanson wrote: > >> I have a list of namedtuples: > >> > >> [{Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='') > >> . . . > >> {Record}(r0=v0, r1=v1,...,r10=v10,r11='',...r93='')] > >> > >> In the first section of code, I process some of the first 10 columns > >> (r0=v0, r1=v1,...,r10=v10), and place the results in blank columns, > >> also in the first 10 columns. > > I too have sometimes started with a namedtuple and then found I needed > > to make changes to the records. I typically abandon namedtuple at this > > point, after only one bad experience trying to work around my choice > > of container. > In a package that I'm working on that enables access natively legacy > C-ISAM files, I've found that I need to create a form of a namedtuple > that makes use of descriptors to access each of the fields within a > record using attribute lookup, but to also create the record object at > runtime using a description of the fields read from another file (C-ISAM > works by using offsets within a record and does not natively store a > mapping between name and offset). I found that using a metaclass to > store the fields in a OrderedDict enabled me to embed a namedtuple to > return the record back to the rest of the package. The code can be found > in pyisam/table/record.py from > http://github.com/rpmoseley/pyisam.git. > The code is written for 3.x but the code path for versions before 3.6 > should work for 2.7. Since I frequently want to work across columns and down rows, sometimes as a step in an overall process, your metaclass sounds like an incredible solution for those sections I need this capability for. However, in my online course studies to date, they only covered classes for two weeks, the last week of which was spent on inheritance. So needless to say, my experience with classes is of the simplest kind, and I'd never heard of metclasses before I started reading this list. But rest assured, the functionality you describe is so potentially useful to me that I will learn what I need to know to understand your metaclass. But this will take some time, and I may not get to it immediately, though I'm saving your message so I can get to it at the first possible chance. Thank you for letting me know about this solution, which I will remember and I look forward to learning how to use it. Deborah From tjreedy at udel.edu Tue Jun 6 17:28:47 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 6 Jun 2017 17:28:47 -0400 Subject: Unhelpful error message In-Reply-To: References: Message-ID: On 6/6/2017 2:29 PM, Chris Angelico wrote: > On Wed, Jun 7, 2017 at 4:22 AM, Skip Montanaro wrote: >> On Tue, Jun 6, 2017 at 12:27 PM, Chris Angelico wrote: >>> Or perhaps showing the repr of the string would be clearer. >> >> This would definitely be better, I think, and require no special >> casing of the empty string. Still, I doubt this would ever be >> backported for fear of breaking code might be out there somewhere >> which relies on the current behavior. At best, it might be added to >> the next 3.x release (3.7?). > > Yeah. Unfortunately this exception doesn't make good use of its args, > so you can't get programmatic information from it (this is true in 3.7 > too actually). So if a change were to be made, I'd recommend putting > the actual string into e.args[1] or something, and then the message > shouldn't need to be parsed. I'm not sure what the backward compat > guarantees on exception messages are, though, so it is possible that > this might be changed in 2.7. Unless a message is erroneous (buggy), it is usually treated as an enhancement. -- Terry Jan Reedy From rosuav at gmail.com Tue Jun 6 17:32:33 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Jun 2017 07:32:33 +1000 Subject: Unhelpful error message In-Reply-To: References: Message-ID: On Wed, Jun 7, 2017 at 7:28 AM, Terry Reedy wrote: >> Yeah. Unfortunately this exception doesn't make good use of its args, >> so you can't get programmatic information from it (this is true in 3.7 >> too actually). So if a change were to be made, I'd recommend putting >> the actual string into e.args[1] or something, and then the message >> shouldn't need to be parsed. I'm not sure what the backward compat >> guarantees on exception messages are, though, so it is possible that >> this might be changed in 2.7. > > > Unless a message is erroneous (buggy), it is usually treated as an > enhancement. That's what I thought. If the OP's happy to upgrade to 3.6/3.7, of course, the problem disappears. ChrisA From ptanyc at gmail.com Tue Jun 6 17:39:54 2017 From: ptanyc at gmail.com (ptanyc at gmail.com) Date: Tue, 6 Jun 2017 14:39:54 -0700 (PDT) Subject: New to Python - Career question Message-ID: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> New to Python and have been at it for about a month now. I'm doing well and like it very much. Considering a career change down the road and have been wondering... What are the job prospects for a middle age entry level programmer. Just trying to get a better understanding where I stand career wise. Appreciate all feed back. Thanks! From tomuxiong at gmx.com Tue Jun 6 17:45:51 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Tue, 6 Jun 2017 14:45:51 -0700 Subject: Unhelpful error message In-Reply-To: References: Message-ID: <2f283981-6d0f-2176-b012-bf2142663f37@gmx.com> On 06/06/2017 02:32 PM, Chris Angelico wrote: > > That's what I thought. If the OP's happy to upgrade to 3.6/3.7, of > course, the problem disappears. > > ChrisA > As far as I can tell this affects all versions of python. Here are some versions I played with: 2.7: >>> float("") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: 3.4: >>> float("") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: 3.6: >>> float("") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: 3.7.0a0: >>> float("") Traceback (most recent call last): File "", line 1, in ValueError: could not convert string to float: Cheers, Thomas From larry.martell at gmail.com Tue Jun 6 18:11:12 2017 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 6 Jun 2017 18:11:12 -0400 Subject: New to Python - Career question In-Reply-To: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> References: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> Message-ID: On Tue, Jun 6, 2017 at 5:39 PM, wrote: > New to Python and have been at it for about a month now. I'm doing well and like it very much. Considering a career change down the road and have been wondering... What are the job prospects for a middle age entry level programmer. Just trying to get a better understanding where I stand career wise. Appreciate all feed back. Thanks! First off, do you have a programming background and/or format training? I run into a lot of people who are, for example, mechanical engineers or some other fairly high tech discipline, and can write code, but have no formal training in it. Usually they write very bad code that is hard to maintain and/or extend. I think twice before hiring anyone like that. Second, what is middle age to you? I am 57 and I have been programming since I was 16 and doing it professionally since I was 20. I have a degree in Software Engineering and I have worked in many different industries and with many different technologies. For the last 5-6 years I have run into a lot of age discrimination, and colleagues of a similar age have told me the same thing. Being interviewed by someone who is 20-30 I can tell they think I am old and they dismiss me right away. These days the only work I seem to get is when I am interviewed by someone my age or older, or if I come with a strong recommendation from someone the person doing the hiring personally knows. In fact I have been told more then once after I have gotten a job and been working at it for a while that the person who hired me was very hesitant to because of my age, but they did because their friend had worked with me before and said very good things. This really worries me because I do not have enough money to retire and doubt I ever will. From greg.ewing at canterbury.ac.nz Tue Jun 6 18:35:37 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 07 Jun 2017 10:35:37 +1200 Subject: Finding the name of an object's source file In-Reply-To: References: Message-ID: Matt Wheeler wrote: >>>>import importlib, os >>>>importlib.import_module(os.path.split.__module__).__file__ > > '/Users/matt/.pyenv/versions/3.6.0/lib/python3.6/posixpath.py' Or just look the module name up in sys.modules, if we know (as we do here) that the module has already been imported. -- Greg From marko at pacujo.net Tue Jun 6 18:37:45 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 07 Jun 2017 01:37:45 +0300 Subject: New to Python - Career question References: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> Message-ID: <87efuwzqrq.fsf@elektro.pacujo.net> ptanyc at gmail.com: > New to Python and have been at it for about a month now. I'm doing > well and like it very much. Considering a career change down the road > and have been wondering... What are the job prospects for a middle age > entry level programmer. Just trying to get a better understanding > where I stand career wise. Appreciate all feed back. Thanks! Different employers hire differently. I have hired several people for my employer, and age has never been a concern. Python is also an important tool where I work. However, the problem in our field is that you have to be quite good to be truly useful. Unfortunately, it seems that only a minority with a formal degree are good enough. On the other hand, I work with some great software developers who don't have a degree at all. One good way to become a good developer and also test oneself is to pick a free software project online a become a contributor. Your commit log entries on GitHub advertise you much better than any pretty-printed r?sum?. Marko From greg.ewing at canterbury.ac.nz Tue Jun 6 18:43:51 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 07 Jun 2017 10:43:51 +1200 Subject: Python program ends with 154 snakes dead; should it continue? In-Reply-To: References: Message-ID: Chris Angelico wrote: > Well, no. Once a Python program ends, it should not continue. At that > point, you're supposed to go back to the OS. Well, the OS is a program, too. -- Greg From larry.martell at gmail.com Tue Jun 6 18:46:49 2017 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 6 Jun 2017 18:46:49 -0400 Subject: New to Python - Career question In-Reply-To: <87efuwzqrq.fsf@elektro.pacujo.net> References: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> <87efuwzqrq.fsf@elektro.pacujo.net> Message-ID: On Tue, Jun 6, 2017 at 6:37 PM, Marko Rauhamaa wrote: > ptanyc at gmail.com: > >> New to Python and have been at it for about a month now. I'm doing >> well and like it very much. Considering a career change down the road >> and have been wondering... What are the job prospects for a middle age >> entry level programmer. Just trying to get a better understanding >> where I stand career wise. Appreciate all feed back. Thanks! > > Different employers hire differently. I have hired several people for my > employer, and age has never been a concern. Python is also an important > tool where I work. > > However, the problem in our field is that you have to be quite good to > be truly useful. Unfortunately, it seems that only a minority with a > formal degree are good enough. On the other hand, I work with some great > software developers who don't have a degree at all. > > One good way to become a good developer and also test oneself is to pick > a free software project online a become a contributor. Your commit log > entries on GitHub advertise you much better than any pretty-printed > r?sum?. I work 70 or more hours a week and have a family and hobbies and a personal life. I do not have time to contribute to any open source projects. And because I cannot show anything on GitHub I have often been summarily disqualified for even getting an interview. (I have donated money to open source projects.) From rosuav at gmail.com Tue Jun 6 18:47:52 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Jun 2017 08:47:52 +1000 Subject: Python program ends with 154 snakes dead; should it continue? In-Reply-To: References: Message-ID: On Wed, Jun 7, 2017 at 8:43 AM, Gregory Ewing wrote: > Chris Angelico wrote: >> >> Well, no. Once a Python program ends, it should not continue. At that >> point, you're supposed to go back to the OS. > > > Well, the OS is a program, too. True, but I don't know of any OS written in Python. I have, however, heard of a Python program that has no OS under it... https://www.youtube.com/watch?v=bYQ_lq5dcvM ... but I don't know where you go when that ends. ChrisA From greg.ewing at canterbury.ac.nz Tue Jun 6 19:09:21 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 07 Jun 2017 11:09:21 +1200 Subject: Access flles on a Android device from Windows PC In-Reply-To: References: Message-ID: Fred Fishbin wrote: > the phone plugged in and > became just "SAMSUNG-SM-G930V", no drive designation, just a Portable Media > Device. ... what do I use for a path? It's probably something like \\SAMSUNG-SM-G930V One way to find out is to open it with Windows Explorer and then click up in the address bar. It should change to show you a textual pathname. -- Greg From steve+python at pearwood.info Tue Jun 6 19:28:53 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 07 Jun 2017 09:28:53 +1000 Subject: Bug or intended behavior? References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: <59373ab6$0$1617$c3e8da3$5496439d@news.astraweb.com> On Wed, 7 Jun 2017 03:16 am, Terry Reedy wrote: >> Personally I prefer a less compact but more explicit alternative: >> >> "foo {}".format(1-2) > > More compact: > >>> f'foo {1-2}' > 'foo -1' Also more mysterious and magical, and not backwards compatible. "Good news everybody! We have something which looks like a string literal but is actually eval() in disguise!" -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From eryksun at gmail.com Tue Jun 6 19:30:37 2017 From: eryksun at gmail.com (eryk sun) Date: Tue, 6 Jun 2017 23:30:37 +0000 Subject: Access flles on a Android device from Windows PC In-Reply-To: References: Message-ID: On Tue, Jun 6, 2017 at 7:36 PM, Fred Fishbin wrote: > > I want to write little program that my friend can run - he'll plug a USB drive > into his Windows 7 PC, plug his phone in a USB port on same PC, then run my > program and it'll xfer some audiobook files over for him. > > I plugged the USB drive in and it became "G:\", but the phone plugged in and > became just "SAMSUNG-SM-G930V", no drive designation, just a Portable Media > Device. Windows Explore can go there so the files are accessible, the phone > isn't looking him out, but what do I use for a path? I tried several different > listdir()'s nothing worked. The shell is probably mounting the device via the Windows Portable Devices (WPD) API. It isn't mounted as a USB disk with a file system, so it won't be assigned a drive letter that you can simply use via file-system APIs such as open (CreateFile), listdir (FindFirstFile), etc. You can look into using WPD Automation via win32com [1]. I haven't used WPD, so I can't offer specific help. https://msdn.microsoft.com/library/dd389295 From jpolo at mail.usf.edu Tue Jun 6 20:58:33 2017 From: jpolo at mail.usf.edu (john polo) Date: Tue, 6 Jun 2017 19:58:33 -0500 Subject: script output appears correct but still raises AssertionError Message-ID: Python People, I am learning about assertions. I wrote a small script that takes 2 inputs, an amino acid sequence and one residue symbol. The script should return what percent of the sequence is residue in output. The point of this script is to use assert for debugging. My script seems to work correctly, but when I use the assert statements that are supposed to test the script, the assertions indicate there is a problem with the script. >>> def aminosleft(sequence, res): ... sequp = sequence.upper() ... lens1 = len(sequp) ... rep = res.upper() ... reps2 = sequp.replace(rep,"") ... lens2 = len(reps2) ... resid = 100* (lens1 - lens2) / lens1 ... print(int(resid)) ... ... >>> aminosleft("MSRSLLLRFLLFLLLLPPLP", "M") 5 >>> assert aminosleft("MSRSLLLRFLLFLLLLPPLP", "M") == 5 5 Traceback (most recent call last): File "", line 1, in AssertionError >>> aminosleft("MSRSLLLRFLLFLLLLPPLP", "r") 10 >>> assert aminosleft("MSRSLLLRFLLFLLLLPPLP", "r") == 10 10 Traceback (most recent call last): File "", line 1, in AssertionError >>> aminosleft("msrslllrfllfllllpplp", "L") 50 >>> assert aminosleft("msrslllrfllfllllpplp", "L") == 50 50 Traceback (most recent call last): File "", line 1, in AssertionError >>> aminosleft("MSRSLLLRFLLFLLLLPPLP", "Y") 0 >>> assert aminosleft("MSRSLLLRFLLFLLLLPPLP", "Y") == 0 0 Traceback (most recent call last): File "", line 1, in AssertionError The script returns an integer. I don't know if the assertion uses an integer or if that matters. What is causing the AssertionError? John From rosuav at gmail.com Tue Jun 6 21:02:15 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Jun 2017 11:02:15 +1000 Subject: script output appears correct but still raises AssertionError In-Reply-To: References: Message-ID: On Wed, Jun 7, 2017 at 10:58 AM, john polo wrote: > I am learning about assertions. I wrote a small script that takes 2 inputs, > an amino acid sequence and one residue symbol. The script should return what > percent of the sequence is residue in output. The point of this script is to > use assert for debugging. My script seems to work correctly, but when I use > the assert statements that are supposed to test the script, the assertions > indicate there is a problem with the script. > > >>>> def aminosleft(sequence, res): > ... sequp = sequence.upper() > ... lens1 = len(sequp) > ... rep = res.upper() > ... reps2 = sequp.replace(rep,"") > ... lens2 = len(reps2) > ... resid = 100* (lens1 - lens2) / lens1 > ... print(int(resid)) > ... > ... > > The script returns an integer. I don't know if the assertion uses an integer > or if that matters. What is causing the AssertionError? No, it doesn't. It's not returning anything (which means it's returning None). Look carefully at the function again, and make it actually return something. ChrisA From frank at chagford.com Wed Jun 7 02:13:44 2017 From: frank at chagford.com (Frank Millman) Date: Wed, 7 Jun 2017 08:13:44 +0200 Subject: Generator and return value Message-ID: Hi all It would be nice to write a generator in such a way that, in addition to 'yielding' each value, it performs some additional work and then 'returns' a final result at the end. >From Python 3.3, anything 'returned' becomes the value of the StopIteration exception, so it is possible, but not pretty. Instead of - my_gen = generator() for item in my_gen(): do_something(item) [how to get the final result?] you can write - my_gen = generator() while True: try: item = next(my_gen()) do_something(item) except StopIteration as e: final_result = e.value Is this the best way to achieve it, or is there a nicer alternative? Thanks Frank Millman From dieter at handshake.de Wed Jun 7 02:53:57 2017 From: dieter at handshake.de (dieter) Date: Wed, 07 Jun 2017 08:53:57 +0200 Subject: Psycopg2 pool clarification References: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> <4cdc7e80393a42f209e8ca0742ace3fa@ravnalaska.net> Message-ID: <87k24ob856.fsf@handshake.de> israel writes: > Since I've gotten no replies to this, I was wondering if someone could > at least confirm which behavior (my expected or my observed) is > *supposed* to be the correct? Should a psycopg2 pool keep connections > open when returned to the pool (if closed is False), or should it > close them as long as there is more than minconn open? i.e is my > observed behavior a bug or a feature? You should ask the author[s] of "psycopg2" about the supposed behavior. >From my point of view, everything depends on the meaning of the "min" and "max" parameters for the pool. You seem to interprete "max" as "keep as many connections as this open". But it can also be a hard limit in the form "never open more than this number of connections". In the latter case, "min" may mean "keep this many connections open at all time". From jussi.piitulainen at helsinki.fi Wed Jun 7 03:09:54 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Wed, 07 Jun 2017 10:09:54 +0300 Subject: Generator and return value References: Message-ID: Frank Millman writes: > It would be nice to write a generator in such a way that, in addition > to 'yielding' each value, it performs some additional work and then > 'returns' a final result at the end. > >> From Python 3.3, anything 'returned' becomes the value of the >> StopIteration > exception, so it is possible, but not pretty. > > Instead of - > my_gen = generator() > for item in my_gen(): > do_something(item) > [how to get the final result?] > > you can write - > my_gen = generator() > while True: > try: > item = next(my_gen()) > do_something(item) > except StopIteration as e: > final_result = e.value > > Is this the best way to achieve it, or is there a nicer alternative? Like this, and imagination is the limit: def generator(box): yield 1 box.append('won') yield 2 box.append('too') yield 3 box.append('tree') mabox = [] for item in generator(mabox): pass print(*mabox) # prints: won too tree From frank at chagford.com Wed Jun 7 03:37:51 2017 From: frank at chagford.com (Frank Millman) Date: Wed, 7 Jun 2017 09:37:51 +0200 Subject: Generator and return value In-Reply-To: References: Message-ID: "Jussi Piitulainen" wrote: > Frank Millman writes: > > > It would be nice to write a generator in such a way that, in addition > > to 'yielding' each value, it performs some additional work and then > > 'returns' a final result at the end. > > > > From Python 3.3, anything 'returned' becomes the value of the > > StopIteration > > exception, so it is possible, but not pretty. > > > > Instead of - > > my_gen = generator() > > for item in my_gen(): > > do_something(item) > > [how to get the final result?] > > > > you can write - > > my_gen = generator() > > while True: > > try: > > item = next(my_gen()) > > do_something(item) > > except StopIteration as e: > > final_result = e.value > > break > > > > Is this the best way to achieve it, or is there a nicer alternative? > > Like this, and imagination is the limit: > > def generator(box): > yield 1 > box.append('won') > yield 2 > box.append('too') > yield 3 > box.append('tree') > > mabox = [] > for item in generator(mabox): pass > print(*mabox) > # prints: won too tree > That is neat. Thanks Frank From steve+python at pearwood.info Wed Jun 7 05:19:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 07 Jun 2017 19:19:38 +1000 Subject: Generator and return value References: Message-ID: <5937c52b$0$1609$c3e8da3$5496439d@news.astraweb.com> On Wed, 7 Jun 2017 05:09 pm, Jussi Piitulainen wrote: > Frank Millman writes: > >> It would be nice to write a generator in such a way that, in addition >> to 'yielding' each value, it performs some additional work and then >> 'returns' a final result at the end. >> >>> From Python 3.3, anything 'returned' becomes the value of the >>> StopIteration >> exception, so it is possible, but not pretty. >> >> Instead of - >> my_gen = generator() >> for item in my_gen(): >> do_something(item) >> [how to get the final result?] Currently, I don't believe there is a way. >> you can write - >> my_gen = generator() >> while True: >> try: >> item = next(my_gen()) >> do_something(item) >> except StopIteration as e: >> final_result = e.value >> >> Is this the best way to achieve it, or is there a nicer alternative? I don't think there are currently any nice alternatives. > Like this, and imagination is the limit: > > def generator(box): > yield 1 > box.append('won') > yield 2 > box.append('too') > yield 3 > box.append('tree') > > mabox = [] > for item in generator(mabox): pass > print(*mabox) > # prints: won too tree Well, that's a little bit better than using a global variable, but it's still pretty ugly, and it doesn't capture the return value so you've answered a completely different question. Here is why I think it is an ugly solution. There are three obvious alternatives to this API: Alternative one: force the caller to provide the "box" argument, whether they care about these extra values or not. for item in generator([]): # argument is ignored, and garbage collected process(item) That's not *too* bad, but still a bit manky. Alternative two: provide a default value for box: def generator(box=[]): ... Pros: now the caller doesn't need to care about box if they don't care about it. Cons: it will leak memory. Every call to generator() with no box argument will collect values in the default list. Alternative three: provide a non-mutable default for box: def generator(box=None): ... Procs: the caller doesn't need to care about box. Cons: writing generator is a lot more complex, you have to check for box is None before appending. There may be clever ways around this, but either way, the complexity of the generator is significantly increased. I'm not saying I'd *never* use this solution. I've used a similar solution myself, treating the argument as a "pass by reference" output parameter, similar to "var" arguments in Pascal. But not often, because it feels ugly. Otherwise, I guess using a while loop is the least-worst existing solution. But here's a neat solution for a feature request: # this doesn't actually work, yet it = generator() for value in it: # run the generator through to completion, as normal process(value) extra_value = it.result where the property lookup it.result: - captures the return value, if the generator has returned; - raise an exception if the generator is still running. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From cs at zip.com.au Wed Jun 7 06:32:18 2017 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 7 Jun 2017 20:32:18 +1000 Subject: Generator and return value In-Reply-To: <5937c52b$0$1609$c3e8da3$5496439d@news.astraweb.com> References: <5937c52b$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20170607103218.GA35253@cskk.homeip.net> On 07Jun2017 19:19, Steve D'Aprano wrote: >> Frank Millman writes: >>> It would be nice to write a generator in such a way that, in addition >>> to 'yielding' each value, it performs some additional work and then >>> 'returns' a final result at the end. >>> >>>> From Python 3.3, anything 'returned' becomes the value of the >>>> StopIteration >>> exception, so it is possible, but not pretty. >>> >>> Instead of - >>> my_gen = generator() >>> for item in my_gen(): >>> do_something(item) >>> [how to get the final result?] > >Currently, I don't believe there is a way. I sometimes yield what would be a return value as the final item. Not very happy with it though. Cheers, Cameron Simpson From mohit_soni87 at ymail.com Wed Jun 7 07:59:29 2017 From: mohit_soni87 at ymail.com (Mohit Soni) Date: Wed, 7 Jun 2017 11:59:29 +0000 (UTC) Subject: Error in initialization of IDLE. In-Reply-To: <1034396071.3466093.1496832486177@mail.yahoo.com> References: <1034396071.3466093.1496832486177.ref@mail.yahoo.com> <1034396071.3466093.1496832486177@mail.yahoo.com> Message-ID: <2071402485.3489485.1496836769029@mail.yahoo.com> Sent from Yahoo Mail on Android I have python 3.5.2 installed and recently I installed python 3.6 and after installing the problem seems to occur. Whenever I start IDLE it shows an error message like "IDLE can't create a sub process or windows firewall might be blocking it" I did a fresh installation of both versions separately (almost all the combinations) but nothing concluded. It's pissing me off since 2 days. Please help me out!! Thank you! From mohit_soni87 at ymail.com Wed Jun 7 07:59:29 2017 From: mohit_soni87 at ymail.com (Mohit Soni) Date: Wed, 7 Jun 2017 11:59:29 +0000 (UTC) Subject: Error in initialization of IDLE. In-Reply-To: <1034396071.3466093.1496832486177@mail.yahoo.com> References: <1034396071.3466093.1496832486177.ref@mail.yahoo.com> <1034396071.3466093.1496832486177@mail.yahoo.com> Message-ID: <2071402485.3489485.1496836769029@mail.yahoo.com> Sent from Yahoo Mail on Android I have python 3.5.2 installed and recently I installed python 3.6 and after installing the problem seems to occur. Whenever I start IDLE it shows an error message like "IDLE can't create a sub process or windows firewall might be blocking it" I did a fresh installation of both versions separately (almost all the combinations) but nothing concluded. It's pissing me off since 2 days. Please help me out!! Thank you! From jorge.conrado at cptec.inpe.br Wed Jun 7 08:20:25 2017 From: jorge.conrado at cptec.inpe.br (jorge.conrado at cptec.inpe.br) Date: Wed, 07 Jun 2017 09:20:25 -0300 Subject: plot time on X axis Message-ID: <26daf54a8682f047ae2db12d2f894ab9@cptec.inpe.br> Hi, I was an IDL user and I'm using Python. I have several meteorological daily time seriee for several years. Please can someone help me. I would like to plot on X axis only the values o the year. Thanks, Conrado From paul.james.barry at gmail.com Wed Jun 7 08:51:55 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Wed, 7 Jun 2017 13:51:55 +0100 Subject: plot time on X axis In-Reply-To: <26daf54a8682f047ae2db12d2f894ab9@cptec.inpe.br> References: <26daf54a8682f047ae2db12d2f894ab9@cptec.inpe.br> Message-ID: Take a look at Jake VanderPlas's book, which is available online as free-to-read: https://github.com/jakevdp/PythonDataScienceHandbook See Chapter 3 (and 4). On 7 June 2017 at 13:20, wrote: > > Hi, > > I was an IDL user and I'm using Python. I have several meteorological > daily time seriee for several years. Please can someone help me. I would > like to plot on X axis only the values o the year. > > Thanks, > > Conrado > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From catalinfest at gmail.com Wed Jun 7 09:08:27 2017 From: catalinfest at gmail.com (blue) Date: Wed, 7 Jun 2017 06:08:27 -0700 (PDT) Subject: how to decrypt encrypted text to a clear text In-Reply-To: <7df3c534-587c-4320-b65e-55a0e9086da6@googlegroups.com> References: <7df3c534-587c-4320-b65e-55a0e9086da6@googlegroups.com> Message-ID: Test again your chain programming way , because : >>> dir(RSA.importKey) ['__call__', '__class__', '__cmp__', '__delattr__', '__doc__', '__format__', '__func__', '__get__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'im_class', 'im_func', 'im_self'] >>> dir(RSA.importKey.decrypt()) Traceback (most recent call last): File "", line 1, in AttributeError: 'function' object has no attribute 'decrypt' On Tuesday, June 6, 2017 at 9:29:52 AM UTC+3, Ho Yeung Lee wrote: > i use wb to write pubic and private key > and succeed to import private, but after decrypt, it return hex number > not a clear text > is there any more key needed? > or do wb influence the result? > > from Crypto.PublicKey import RSA > keypair = RSA.generate(2048) > alice_privkey = keypair.exportKey('PEM', 'mysecret', pkcs=1) > #alice_privkey = keypair.exportKey() > alice_pubkey = keypair.publickey().exportKey() > > text_file = open("alice_pubkey.txt", "wb") > text_file.write(alice_pubkey) > text_file.close() > > keypair = RSA.generate(2048) > bob_privkey = keypair.exportKey('PEM', 'mysecret2', pkcs=1) > #bob_privkey = keypair.exportKey() > bob_pubkey = keypair.publickey().exportKey() > > text_file = open("bob_pubkey.txt", "wb") > text_file.write(bob_pubkey) > text_file.close() > > text_file = open("alice_privkey.pem", "wb") > text_file.write(alice_privkey) > text_file.close() > text_file = open("bob_privkey.pem", "wb") > text_file.write(bob_privkey) > text_file.close() > > > #step 2 > #use alice public key to encrypt > pubkey = RSA.importKey(alice_pubkey) > alice_masterkey = pubkey.encrypt("i am Martin", None) > > text_file = open("alice_masterkey.txt", "w") > text_file.write(bob_pubkey) > text_file.close() > > quit() > python > text_file = open("alice_masterkey.txt", "r") > alice_masterkey=text_file.read() > text_file.close() > text_file = open("alice_privkey.pem", "r") > alice_privkey=text_file.read() > text_file.close() > text_file = open("alice_pubkey.txt", "r") > alice_pubkey=text_file.read() > text_file.close() > > from Crypto.PublicKey import RSA > pubkey = RSA.importKey(alice_pubkey) > privkey = RSA.importKey(alice_privkey,passphrase="mysecret") > encryption_key = privkey.decrypt(alice_masterkey) > encryption_key > > quit() > python > text_file = open("alice_masterkey.txt", "r") > alice_masterkey=text_file.read() > text_file.close() > text_file = open("alice_privkey.pem", "r") > alice_privkey=text_file.read() > text_file.close() > text_file = open("alice_pubkey.txt", "r") > alice_pubkey=text_file.read() > text_file.close() > > from Crypto.PublicKey import RSA > pubkey = RSA.importKey(alice_pubkey) > privkey = RSA.importKey(alice_privkey,passphrase="mysecret") > encryption_key = privkey.decrypt(alice_masterkey) > encryption_key > > >>> encryption_key > 'o\x94\xaeC\xe0S\x81\x05t\xd8\\A\x10?\xd2\xe5\x8c5\xc9\x1d\x14\xc7\xfd)Cs\x8b"cg\x16y\xe2\xf2L\xf1-\x08qHt\x99\xbc\xb5\xf6_\x17c\xd2&Z\x0b\xc5t\t\xe0\x8b\x03G\x10\xce\xd6\xcd\x86\xfc!\xc9i\xa2\xab\x9d\x8a\x92\xfc7 g\xa5$\x91\x85\xa2L]I\xd6\xc6\xaez\xed\x01\x95\xee)8z\x18\xc9aag\x97\x97\xb0\\)\xec"\xe4\xbez\xd3\xa8\'k%\x12\x1d\xf9\xf0\x0e\x0c\xcb\xa8\xb1\xe7}\x90\xd3\xcfs@\xc2m\x1a^\x1b0\xa7\xdd\xcd\xea\x1f\xd5\x08\x13+y"]vu\xe3\x9e\xba\x97\x10\x90S\xea\xae1=r4Yp,\xe3\xa9\xc66H\xa7\x95[M|n\x91\x98\x9c,\xc4\xf5\x7f\x8cJ\x03\xba\x04Z0lV\xe1\xd6d\xeec@\xe1\xa0\xec\x81]\xef5\r\x12\x88\xbe/\xfc\xe01\xaacn,\x8a\xe1\x14\x8a\xf4\xd85\xd8\xabD\x137\xe7T\xc4\xc1\x84b.\xd9RZ\x0e\x03#\x1e\x8dl\xe8\xe4N^\r\xf0\x1d\x8c' From jpolo at mail.usf.edu Wed Jun 7 09:19:56 2017 From: jpolo at mail.usf.edu (john polo) Date: Wed, 7 Jun 2017 08:19:56 -0500 Subject: script output appears correct but still raises, AssertionError In-Reply-To: References: Message-ID: <92bfab62-ae04-b833-7abd-3eea660f5cdd@mail.usf.edu> ChrisA, Thank you for pointing out my error: using print() when I should have used return(). John From rgaddi at highlandtechnology.invalid Wed Jun 7 12:00:36 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 7 Jun 2017 09:00:36 -0700 Subject: Generator and return value In-Reply-To: References: Message-ID: On 06/06/2017 11:13 PM, Frank Millman wrote: > Hi all > > It would be nice to write a generator in such a way that, in addition to > 'yielding' each value, it performs some additional work and then > 'returns' a final result at the end. > >> From Python 3.3, anything 'returned' becomes the value of the >> StopIteration > exception, so it is possible, but not pretty. > > Instead of - > my_gen = generator() > for item in my_gen(): > do_something(item) > [how to get the final result?] > > you can write - > my_gen = generator() > while True: > try: > item = next(my_gen()) > do_something(item) > except StopIteration as e: > final_result = e.value > > Is this the best way to achieve it, or is there a nicer alternative? > > Thanks > > Frank Millman > > class MyGeneration: def __iter__(self): yield from ('people', 'try', 'to', 'put', 'us', 'down') self.final = 'talking about' mygen = MyGeneration() for item in mygen: print(item) print(mygen.final) -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From pkpearson at nowhere.invalid Wed Jun 7 12:27:55 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 7 Jun 2017 16:27:55 GMT Subject: Bug or intended behavior? References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On Tue, 6 Jun 2017 13:16:00 -0400, Terry Reedy wrote: > On 6/5/2017 1:01 PM, Peter Pearson wrote: >> On Fri, 2 Jun 2017 10:17:05 -0700 (PDT), sean.dizazzo at gmail.com wrote: >> [snip] >>>>>> print "foo %s" % 1-2 >>> Traceback (most recent call last): >>> File "", line 1, in >>> TypeError: unsupported operand type(s) for -: 'str' and 'int' >> >> Others have already pointed out that you're assuming the >> wrong precedence: >> >> Say >> "foo %s" % (1-2) >> not >> ("foo %s" % 1) - 2 >> . >> >> Personally I prefer a less compact but more explicit alternative: >> >> "foo {}".format(1-2) > > More compact: > >>> f'foo {1-2}' > 'foo -1' Sarcastic thanks, dude. Excuse me while I scrub my screen with Clorox. Or maybe my eyeballs. More seriously, I thought "format" was the Cool New Thing toward which all the cool kids were moving. But here I tried to be cool and put in a plug for "format", and the hip community seems to be sticking up for "%". Can I never get with the times? -- To email me, substitute nowhere->runbox, invalid->com. From israel at ravnalaska.net Wed Jun 7 12:42:04 2017 From: israel at ravnalaska.net (israel) Date: Wed, 07 Jun 2017 08:42:04 -0800 Subject: Psycopg2 pool clarification In-Reply-To: <87k24ob856.fsf@handshake.de> References: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> <4cdc7e80393a42f209e8ca0742ace3fa@ravnalaska.net> <87k24ob856.fsf@handshake.de> Message-ID: <9d535a63e0013a93d741d097815c254a@ravnalaska.net> On 2017-06-06 22:53, dieter wrote: > israel writes: >> Since I've gotten no replies to this, I was wondering if someone could >> at least confirm which behavior (my expected or my observed) is >> *supposed* to be the correct? Should a psycopg2 pool keep connections >> open when returned to the pool (if closed is False), or should it >> close them as long as there is more than minconn open? i.e is my >> observed behavior a bug or a feature? > > You should ask the author[s] of "psycopg2" about the supposed behavior. > > > From my point of view, everything depends on the meaning of the "min" > and "max" parameters for the pool. > > You seem to interprete "max" as "keep as many connections as this > open". > But it can also be a hard limit in the form "never open more than this > number of connections". In the latter case, "min" may mean "keep this > many connections open at all time". You are right about my interpretation of "max", and also about the actual meaning. Thus the reason I was asking :-). I did post on the bug report forum, and was informed that the observed behavior was the correct behavior. As such, using psycopg2's pool is essentially worthless for me (plenty of use for it, i'm sure, just not for me/my use case). So let me ask a different, but related, question: Is there a Python library available that gives me the behavior I described in my first post, where connections are "cached" for future use for a time? Or should I just write my own? I didn't find anything with some quick googling, other than middleware servers like pgpool which, while they have the behavior I want (at least from my reading), will still require the overhead of making a connection (perhaps less than direct to postgres? Any performance comparisons out there?), not to mention keeping yet another service configured/running. I would prefer to keep the pool internal to my application, if possible, and simply reuse existing connections rather than making new ones. Thanks! From pkpearson at nowhere.invalid Wed Jun 7 12:56:31 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 7 Jun 2017 16:56:31 GMT Subject: plot time on X axis References: <26daf54a8682f047ae2db12d2f894ab9@cptec.inpe.br> Message-ID: On Wed, 07 Jun 2017 09:20:25 -0300, jorge.conrado at cptec.inpe.br wrote: [snip] > I was an IDL user and I'm using Python. I have several meteorological > daily time seriee for several years. Please can someone help me. I would > like to plot on X axis only the values o the year. Is matplotlib allowed? Example (Python 2.7)(Note that most of this example is just parsing data, not plotting:): from matplotlib import pyplot as plt from datetime import date import re dwcd = """ 2014-06-01: 82.00% ( 1) 89.50% ( 2) 39.00% ( 1) 0.259 2014-07-01: 100.00% ( 1) 89.00% ( 3) 0.00% ( 1) 0.264 2014-08-01: 79.50% ( 2) 85.50% ( 4) 53.00% ( 1) 0.273 2014-09-01: 85.00% ( 3) 98.00% ( 6) 87.00% ( 3) 0.495 2014-10-01: 86.00% ( 7) 97.00% (10) 82.50% ( 4) 0.553 2014-11-01: 93.50% (10) 98.50% (10) 39.00% ( 6) 0.215 2014-12-01: 97.00% (10) 100.00% (10) 66.50% ( 6) 0.025 2015-01-01: 72.50% (12) 94.00% (11) 39.00% ( 6) 0.025 2015-02-01: 66.00% (12) 88.50% (12) 58.50% ( 8) 0.248 2015-03-01: 79.00% (15) 95.50% (12) 77.00% ( 9) 0.360 2015-04-01: 87.00% (15) 95.50% (12) 68.00% ( 9) 0.039 2015-05-01: 85.50% (18) 90.00% (12) 87.00% ( 9) 0.479 """ def get_time(r): r = r.strip() return date(year=int(r[0:4]), month=int(r[5:7]), day=int(r[8:10])) def get_field(r, field_name): m = re.match(r"(?P[^:]+): +" r"(?P[0-9.]+)% +" r"\( *(?P[0-9]+)\) +" r"(?P[0-9.]+)% +" r"\( *(?P[0-9]+)\) +" r"(?P[0-9.]+)% +" r"\( *(?P[0-9]+)\) +" r"(?P

[0-9.]+)", r.strip()) return m.group(field_name) x = [get_time(r) for r in dwcd.split("\n") if r] y_a = [float(get_field(r, "value_a")) for r in dwcd.split("\n") if r] y_b = [float(get_field(r, "value_b")) for r in dwcd.split("\n") if r] y_c = [float(get_field(r, "value_c")) for r in dwcd.split("\n") if r] plt.plot(x, y_a, color="red", label="Group A") plt.plot(x, y_b, color="green", label="Group B") plt.plot(x, y_c, color="blue", label="Group C") plt.plot(date(2015,5,20), 101, marker="x", color="white") plt.ylabel("Y label") plt.legend(loc="upper left") fig = plt.gcf() fig.autofmt_xdate() plt.show() -- To email me, substitute nowhere->runbox, invalid->com. From skip.montanaro at gmail.com Wed Jun 7 13:16:00 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 7 Jun 2017 12:16:00 -0500 Subject: Bug or intended behavior? In-Reply-To: References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On Wed, Jun 7, 2017 at 11:27 AM, Peter Pearson wrote: > I thought "format" was the Cool New Thing toward which > all the cool kids were moving. But here I tried to be cool and put in a > plug for "format", and the hip community seems to be sticking up for > "%". The f"..." string is pretty new. Only the hippest of the hip (>= Python 3.6) are using it at this point. I only recently started using "...".format() to ease some anticipated far-in-the-future switch to Python 3 for some of my work. It's grown on me a bit. As an old C programmer, I've been pretty married to printf-style formatting. Skip From rosuav at gmail.com Wed Jun 7 13:29:51 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 8 Jun 2017 03:29:51 +1000 Subject: Bug or intended behavior? In-Reply-To: References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> Message-ID: On Thu, Jun 8, 2017 at 2:27 AM, Peter Pearson wrote: > More seriously, I thought "format" was the Cool New Thing toward which > all the cool kids were moving. But here I tried to be cool and put in a > plug for "format", and the hip community seems to be sticking up for > "%". Can I never get with the times? The times aren't moving. Both percent formatting and .format are here to stay. Python is not JavaScript, and you don't have to "move with the times"; we don't have eight major versions in seven years, or four in five years, or ten in... I don't know how many years, but it's an npm-installable module, so not more than seven or eight. ChrisA From neilc at norwich.edu Wed Jun 7 13:35:59 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Wed, 7 Jun 2017 17:35:59 +0000 (UTC) Subject: Namedtuple problem #32.11.d References: <004601d2df00$656e09b0$2860e747@sambora> Message-ID: On 2017-06-06, Deborah Swanson wrote: >> I too have sometimes started with a namedtuple and then found >> I needed to make changes to the records. I typically abandon >> namedtuple at this point, after only one bad experience trying >> to work around my choice of container. > > I can appreciate that reaction. > > Guess I'm a bit of a bulldog though (right ot wrong), and the > concept of namedtuples is so ideally suited for the Excel > spreadsheet conversions I'm working on, I'll keep on pushing > the boundaries to see how they can be made to work. ;) The namedtuple has found a happy place in my repertoire as the return value of functions that transform external read-only tabular data into a convenient form for lookup. I agree pushing a language feature beyond its preferable use cases is a good way to learn concepts and illuminate dark corners of both my own skill and Python's features. An Excel spreadsheet that represents a table of data is fairly simple to map onto a Python dict. One nearly codeless way is to export it from Excel as a csv file and then read it with csv.DictReader. -- Neil Cerutti From ian.g.kelly at gmail.com Wed Jun 7 13:58:09 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 7 Jun 2017 11:58:09 -0600 Subject: Generator and return value In-Reply-To: References: Message-ID: On Wed, Jun 7, 2017 at 10:00 AM, Rob Gaddi wrote: > > On 06/06/2017 11:13 PM, Frank Millman wrote: >> >> Hi all >> >> It would be nice to write a generator in such a way that, in addition to 'yielding' each value, it performs some additional work and then 'returns' a final result at the end. >> >>> From Python 3.3, anything 'returned' becomes the value of the StopIteration >> >> exception, so it is possible, but not pretty. >> >> Instead of - >> my_gen = generator() >> for item in my_gen(): >> do_something(item) >> [how to get the final result?] >> >> you can write - >> my_gen = generator() >> while True: >> try: >> item = next(my_gen()) >> do_something(item) >> except StopIteration as e: >> final_result = e.value >> >> Is this the best way to achieve it, or is there a nicer alternative? >> >> Thanks >> >> Frank Millman > > class MyGeneration: > def __iter__(self): > yield from ('people', 'try', 'to', 'put', 'us', 'down') > self.final = 'talking about' > > mygen = MyGeneration() > for item in mygen: > print(item) > print(mygen.final) Or as a generic wrapper: class CaptureValue: def __init__(self, iterable): self._iter = iter(iterable) def __iter__(self): return self def __next__(self): try: return next(self._iter) except StopIteration as e: self.value = e.value raise capture = CaptureValue(generator()) print(list(capture)) print(capture.value) From python at deborahswanson.net Wed Jun 7 16:19:11 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Wed, 7 Jun 2017 13:19:11 -0700 Subject: !RE: Namedtuple problem #32.11.d In-Reply-To: Message-ID: <007f01d2dfcb$53e5cd10$2860e747@sambora> Neil Cerutti wrote, on Wednesday, June 07, 2017 10:36 AM > > On 2017-06-06, Deborah Swanson wrote: > >> I too have sometimes started with a namedtuple and then found I > >> needed to make changes to the records. I typically abandon > namedtuple > >> at this point, after only one bad experience trying to > work around my > >> choice of container. > > > > I can appreciate that reaction. > > > > Guess I'm a bit of a bulldog though (right ot wrong), and the concept > > of namedtuples is so ideally suited for the Excel spreadsheet > > conversions I'm working on, I'll keep on pushing the boundaries to see > > how they can be made to work. ;) > > The namedtuple has found a happy place in my repertoire as > the return value of functions that transform external > read-only tabular data into a convenient form for lookup. > > I agree pushing a language feature beyond its preferable use > cases is a good way to learn concepts and illuminate dark > corners of both my own skill and Python's features. I certainly have learned a lot by doing exactly that. Sometimes it isn't so much people giving the solution to a problem, though that's definitely to the good, but the alternate solutions that are proposed can also be highly instructive. > An Excel spreadsheet that represents a table of data is > fairly simple to map onto a Python dict. One nearly codeless > way is to export it from Excel as a csv file and then read it > with csv.DictReader. > > -- > Neil Cerutti csv.DictReader! I didn't know there was one! I've been thinking about how a spreadsheet could be put into a dict, but wasn't quite coming up with a good way. But a csv.DictReader would be perfect, and I imagine the keys would be the column names, which is also perfect. Thanks for the lead on csv.DictReader. I suppose if I'd known one existed it would be easy to find, but when you don't know, and you don't think of it... Deborah From tjreedy at udel.edu Wed Jun 7 17:06:07 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 7 Jun 2017 17:06:07 -0400 Subject: Error in initialization of IDLE. In-Reply-To: <2071402485.3489485.1496836769029@mail.yahoo.com> References: <1034396071.3466093.1496832486177.ref@mail.yahoo.com> <1034396071.3466093.1496832486177@mail.yahoo.com> <2071402485.3489485.1496836769029@mail.yahoo.com> Message-ID: On 6/7/2017 7:59 AM, Mohit Soni via Python-list wrote: > I have python 3.5.2 installed and recently I installed python 3.6 and after installing the problem seems to occur. What OS are you using? Did IDLE run correctly before adding 3.6? How did you start IDLE? > Whenever I start IDLE it shows an error message like "IDLE can't create a sub process or windows firewall might be blocking it" See https://bugs.python.org/issue25514#msg258498 for at least 7 possible causes. -- Terry Jan Reedy From subhabangalore at gmail.com Wed Jun 7 17:57:28 2017 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Wed, 7 Jun 2017 14:57:28 -0700 (PDT) Subject: Time Calculation to Tag a Sentence/File Message-ID: I am trying to calculate the time required to tag one sentence/file by one trained NLTK HMM Tagger. To do this I am writing the following code, please suggest if I need to revise anything here. import nltk from nltk.corpus.reader import TaggedCorpusReader import time #HMM reader = TaggedCorpusReader('/python27/', r'.*\.pos') f1=reader.fileids() print f1 sents=reader.tagged_sents() ls=len(sents) print "Total No of Sentences:",ls train_sents=sents[0:40] test_sents=sents[41:46] #TRAINING & TESTING hmm_tagger=nltk.HiddenMarkovModelTagger.train(train_sents) test=hmm_tagger.test(test_sents) appli_sent1=reader.sents(fileids='minicv.pos')[0] print "SAMPLE INPUT:",appli_sent1 #TIME CALCULATION start_time = time.clock() application=hmm_tagger.tag(appli_sent1) #I MAY REPLACE WITH ONE DOCUMENT print "ENTITY RECOGNIZED",application print "Time Taken Is:",time.clock() - start_time, "seconds" NB: This is a toy kind example and I did not follow much of training/testing size parameters. My question is only for the time calculation part. It is not a forum for Machine Learning, but as there are many people who has very high level knowledge on it, any one is most welcome to give his/her valuable feedback which may improve my knowledge. As the code is pasted here from IDLE (with Python2.7 on MS-Windows 7) I could not maintain proper indentation, apology for the same. From cinthyab at ldsbc.edu Wed Jun 7 19:56:23 2017 From: cinthyab at ldsbc.edu (CB) Date: Wed, 7 Jun 2017 16:56:23 -0700 (PDT) Subject: Hello from a super noob! Message-ID: <3da626ed-e170-4903-933c-5fde0d2338cc@googlegroups.com> Hi everyone, I am taking a python class and I'm stuck in an exercise. what am i doing wrong? Can anyone try to run it? Thanks so much! #Description:Input validation and while loops. import random def main(): #main function need in all programs for automated testing #your program goes here print() print("This program will help us practice input validation and while loops.") print("The user will be asked to enter two numbers which will both be validated. ") print("The sum of the numbers will then be displayed in a complex print statement ") print("and the user will be asked if they would like to run the program again." ) print() print() while True: FirstNumber = input ("Please enter the first number: ") if FirstNumber.isdigit (): FirstNumber = int(FirstNumber) break else: print ("Invalid response. Please enter a whole number. " ) while True: SecondNumber = input ("Please enter the second number: " ) if SecondNumber.isdigit(): SecondNumber= int(SecondNumber) break else: print("Invalid response. Please enter a whole number." ) print() print (str(FirstNumber) + " + " + str(SecondNumber)+ " = " + str(FirstNumber + SecondNumber)) print() while True: ans= input('Would you like to run the program again (Y/N) : ') if ans== 'Y' or ans== 'N': break else: print(" lnvalid response. Please answer with 'Y' or 'N' ") if ans== 'N': break From python at mrabarnett.plus.com Wed Jun 7 20:34:56 2017 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 8 Jun 2017 01:34:56 +0100 Subject: Hello from a super noob! In-Reply-To: <3da626ed-e170-4903-933c-5fde0d2338cc@googlegroups.com> References: <3da626ed-e170-4903-933c-5fde0d2338cc@googlegroups.com> Message-ID: On 2017-06-08 00:56, CB wrote: > Hi everyone, > I am taking a python class and I'm stuck in an exercise. > > what am i doing wrong? Can anyone try to run it? Thanks so much! > > #Description:Input validation and while loops. > > > import random > def main(): #main function need in all programs for automated testing > > > #your program goes here > > print() > > > > > print("This program will help us practice input validation and while loops.") > print("The user will be asked to enter two numbers which will both be validated. ") > print("The sum of the numbers will then be displayed in a complex print statement ") > print("and the user will be asked if they would like to run the program again." > ) > print() > print() > > while True: > FirstNumber = input ("Please enter the first number: ") > if FirstNumber.isdigit (): > FirstNumber = int(FirstNumber) > break > else: > print ("Invalid response. Please enter a whole number. " ) > > while True: > > SecondNumber = input ("Please enter the second number: " ) > if SecondNumber.isdigit(): > SecondNumber= int(SecondNumber) > > break > else: > print("Invalid response. Please enter a whole number." ) > > print() > print (str(FirstNumber) + " + " + str(SecondNumber)+ " = " + str(FirstNumber + SecondNumber)) > print() > > while True: > > ans= input('Would you like to run the program again (Y/N) : ') > if ans== 'Y' or ans== 'N': > break > > else: > print(" lnvalid response. Please answer with 'Y' or 'N' ") > > if ans== 'N': > break > > You haven't said what the problem is. It looks OK, apart from the indentation, which is important to get right in Python. Also, you've defined a function 'main' but not called it, and imported a module but not used it, which is pointless. From ptanyc at gmail.com Wed Jun 7 20:37:09 2017 From: ptanyc at gmail.com (Matt) Date: Wed, 7 Jun 2017 17:37:09 -0700 (PDT) Subject: New to Python - Career question In-Reply-To: <87efuwzqrq.fsf@elektro.pacujo.net> References: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> <87efuwzqrq.fsf@elektro.pacujo.net> Message-ID: <63f80fc4-b75f-4f47-ba19-8f4caaa3fe21@googlegroups.com> On Tuesday, June 6, 2017 at 3:37:56 PM UTC-7, Marko Rauhamaa wrote: > ptanyc at gmail.com: > > > New to Python and have been at it for about a month now. I'm doing > > well and like it very much. Considering a career change down the road > > and have been wondering... What are the job prospects for a middle age > > entry level programmer. Just trying to get a better understanding > > where I stand career wise. Appreciate all feed back. Thanks! > > Different employers hire differently. I have hired several people for my > employer, and age has never been a concern. Python is also an important > tool where I work. > > However, the problem in our field is that you have to be quite good to > be truly useful. Unfortunately, it seems that only a minority with a > formal degree are good enough. On the other hand, I work with some great > software developers who don't have a degree at all. > > One good way to become a good developer and also test oneself is to pick > a free software project online a become a contributor. Your commit log > entries on GitHub advertise you much better than any pretty-printed > r?sum?. > > > Marko Marko, Thanks, appreciate your input. I'll check out Github and follow your suggestions. From steve+python at pearwood.info Wed Jun 7 21:35:14 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 08 Jun 2017 11:35:14 +1000 Subject: Hello from a super noob! References: <3da626ed-e170-4903-933c-5fde0d2338cc@googlegroups.com> Message-ID: <5938a9d3$0$1595$c3e8da3$5496439d@news.astraweb.com> On Thu, 8 Jun 2017 09:56 am, CB wrote: > Can anyone try to run it? Yes, you can. Doctor to patient: "So, what seems to be the problem?" Patient: "You're the doctor, you tell me." -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From FredFishbin at hotmail.com Wed Jun 7 23:41:20 2017 From: FredFishbin at hotmail.com (Fred Fishbin) Date: Wed, 07 Jun 2017 22:41:20 -0500 Subject: Access flles on a Android device from Windows PC << YES! Thanks References: Message-ID: Yup, that seems to be the deal, and there doesn't seem tpo be a really simple way to deal with this. ...but at least I know what I need to look for. thanks! Freddie eryk sun wrote: >On Tue, Jun 6, 2017 at 7:36 PM, Fred Fishbin wrote: >> >> I want to write little program that my friend can run - he'll plug a USB >>drive >> into his Windows 7 PC, plug his phone in a USB port on same PC, then run my >> program and it'll xfer some audiobook files over for him. >> >> I plugged the USB drive in and it became "G:\", but the phone plugged in and >> became just "SAMSUNG-SM-G930V", no drive designation, just a Portable Media >> Device. Windows Explore can go there so the files are accessible, the phone >> isn't looking him out, but what do I use for a path? I tried several >>different >> listdir()'s nothing worked. > >The shell is probably mounting the device via the Windows Portable >Devices (WPD) API. It isn't mounted as a USB disk with a file system, >so it won't be assigned a drive letter that you can simply use via >file-system APIs such as open (CreateFile), listdir (FindFirstFile), >etc. You can look into using WPD Automation via win32com [1]. I >haven't used WPD, so I can't offer specific help. > >https://msdn.microsoft.com/library/dd389295 From dieter at handshake.de Thu Jun 8 02:31:06 2017 From: dieter at handshake.de (dieter) Date: Thu, 08 Jun 2017 08:31:06 +0200 Subject: Psycopg2 pool clarification References: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> <4cdc7e80393a42f209e8ca0742ace3fa@ravnalaska.net> <87k24ob856.fsf@handshake.de> <9d535a63e0013a93d741d097815c254a@ravnalaska.net> Message-ID: <87bmpzxa6t.fsf@handshake.de> israel writes: > On 2017-06-06 22:53, dieter wrote: > ... > As such, using psycopg2's pool is essentially > worthless for me (plenty of use for it, i'm sure, just not for me/my > use case). Could you not simply adjust the value for the "min" parameter? If you want at least "n" open connections, then set "min" to "n". From fabiocorvinoe73779 at gmail.com Thu Jun 8 04:18:40 2017 From: fabiocorvinoe73779 at gmail.com (ilCorvo) Date: Thu, 8 Jun 2017 01:18:40 -0700 (PDT) Subject: How to use win32api package to send to printer a file in memory Message-ID: I have an Angular application that is communicating via websocket with a python app. Actually, I save the file in Angular, then I pass to the websocket the file path and I print it by win32api.ShellExecute. Anyway, I do not really need to save the file, so would be much better to send the content though the websocket and then then print it. Can anyone send me some code snippet to show: 1. Which is the best data structure to receive the file content via websocket (bytes array?) 2. How to invoke win32api.ShellExecute, or equivalent function, to print the content. Thank you From chris at withers.org Thu Jun 8 04:25:29 2017 From: chris at withers.org (Chris Withers) Date: Thu, 8 Jun 2017 09:25:29 +0100 Subject: testfixtures 5.1.0 released! Message-ID: <09c5f606-20c5-9766-f66e-dc93da102934@withers.org> Hi All, I'm pleased to announce the release of testfixtures 5.1.0 featuring the following: * Added support for including non-editable fields to the|comparer|used by|compare()|when comparing/django/|Model|instances. The package is on PyPI and a full list of all the links to docs, issue trackers and the like can be found here: https://github.com/Simplistix/testfixtures Any questions, please do ask on the Testing in Python list or on the Simplistix open source mailing list... cheers, Chris From phnetynka at gmail.com Thu Jun 8 04:58:27 2017 From: phnetynka at gmail.com (phnetynka at gmail.com) Date: Thu, 8 Jun 2017 01:58:27 -0700 (PDT) Subject: CfP: 14th International Conference on Managed Languages & Runtimes Message-ID: <2dcea172-f1cb-4dd7-b5d2-8c6c62b7dd98@googlegroups.com> CALL FOR PAPERS - ManLang '17 14th International Conference on Managed Languages & Runtimes (ManLang, formerly PPPJ) September 25-29, 2017, Prague, Czech Republic http://d3s.mff.cuni.cz/conferences/manlang17/ Sponsored by Charles University and Oracle Labs In-cooperation with ACM SIGAPP and SIGPLAN ManLang is a premier forum for presenting and discussing innovations and breakthroughs in the area of programming languages and runtime systems, which form the basis of many modern computing systems, from small scale (embedded and real-time systems) to large-scale (cloud-computing and big-data platforms). ------------------------------------------------------------------------------- TOPICS - Virtual machines * Managed runtime systems (JVM, Dalvik VM and Android Runtime (ART), LLVM, .NET CLR, RPython, etc.) * VM design and optimization * VMs for mobile and embedded devices * VMs for real-time applications * Isolation and resource control * Memory management - Languages * Managed languages (Java, Scala, JavaScript, Python, Ruby, C#, F#, Clojure, Groovy, Kotlin, R, Smalltalk, Racket, etc.) * Domain-specific languages * Language design and calculi * Compilers * Language interoperability * Parallelism and concurrency * Modular and aspect-oriented programming * Model-driven development * Frameworks and applications * Teaching - Techniques and tools * Static and dynamic program analysis * Real-time systems * Embedded systems * Testing * Verification * Monitoring and debugging * Security and information flow * Workload characterization and performance evaluation Do not hesitate to contact the Program Chair (Shigeru Chiba ) to clarify if a particular topic falls within the scope of ManLang '17. ------------------------------------------------------------------------------- IMPORTANT DATES Abstract submission deadline: June 23, 2017 Submission deadline: June 26, 2017 Author notification: August 7, 2017 Camera-ready papers deadline: August 21, 2017 ------------------------------------------------------------------------------- SUBMISSIONS ManLang '17 accepts four types of submissions: - Regular research paper: up to 12 pages - Work-in-progress paper: up to 6 pages - Industry and tool paper: up to 6 pages - Poster (standalone or accompanying paper submission) The conference proceedings will be published as part of the ACM International Conference Proceedings Series and will be disseminated through the ACM Digital Library. Research papers will be judged on their relevance, novelty, technical rigor, and contribution to the state-of-the-art. For work-in-progress research papers, more emphasis will be placed on novelty and the potential of the new idea than on technical rigor and experimental results. Industry and tool papers will be judged on their relevance, usefulness, and results. Suitability for demonstration and availability will also be considered for tool papers. Posters can accompany any submission, in particular to provide additional demonstration and discussion opportunities. Criteria for standalone posters will be similar to criteria for short papers. ------------------------------------------------------------------------------- PROGRAM CHAIR * Shigeru Chiba, University of Tokyo, Japan GENERAL CHAIR * Petr Tuma, Charles University, Czech Republic PROGRAM COMMITTEE * Walter Binder, University of Lugano (USI), Switzerland * Christoph Bockisch, University Marburg, Germany * Elisa Gonzalez Boix, Vrije Universiteit Brussel, Belgium * Carl Friedrich Bolz, PyPy * Walter Cazzola, Universit? degli Studi di Milano, Italy * Yvonne Coady, University of Victoria, Canada * St?phane Ducasse, Inria Lille Nord Europe, France * G?rel Hedin, Lunds University, Sweden * Robert Hirschfeld, HPI, Germany * Tony Hosking, Purdue University, USA * Doug Lea, State University of New York at Oswego, USA * J. Eliot Moss, University of Massachusetts Amherst, USA * Hanspeter M?ssenb?ck, Johannes Kepler University Linz, Austria * Tiark Rompf, Purdue University, USA * Koichi Sasada, Cookpad Inc., Japan * Martin Schoeberl, Technical University of Denmark * Jeremy Singer, University of Glasgow, Scotland * Vincent St-Amour, Northwestern University, USA * Laurence Tratt, King's College London, UK * Jan Vitek, Northeastern University, USA * Christian Wimmer, Oracle Labs, USA * Jianjun Zhao, Kyushu University, Japan From guillaume.paulet at giome.fr Thu Jun 8 05:46:41 2017 From: guillaume.paulet at giome.fr (guillaume.paulet at giome.fr) Date: Thu, 08 Jun 2017 11:46:41 +0200 Subject: =?UTF-8?Q?New_release_of_Scalpl_=28v0=2E2=2E4=29_=E2=9C=A8?= =?UTF-8?Q?=F0=9F=8D=B0=E2=9C=A8?= Message-ID: Good morning evernyone ! I released a new version (0.2.4) of Scalpl yesterday evening which is available on PyPI :) https://github.com/ducdetronquito/scalpl https://pypi.python.org/pypi/scalpl/ Scalpl is a lightweight wrapper that helps you to operate on nested dictionaries through the built-in dict API, by using dot-separated string keys. It's not a drop-in replacement for your dictionnaries, just syntactic sugar to avoid this['annoying']['kind']['of']['things'] and prefer['a.different.approach']. It aims to keep the power of the standard dict API while being lighter and faster that Box or Addict. This new release allows you to traverse nested list in your dictionnaries: ``` from scalpl import Cut data = {...} proxy = Cut(Data) proxy.update({'users[42][0].skills', 'Python'}) # data['users'][42][0]['skills'] == 'Python ``` It also contains: * A pretty good refactoring of the code base. * Better exceptions * More tests I also tried to improve the README, with a benchmark section and a FAQ. I would really appreciate your feedbacks to improve this project ! Have a great day :) ??? From rhodri at kynesim.co.uk Thu Jun 8 07:28:31 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 8 Jun 2017 12:28:31 +0100 Subject: Hello from a super noob! In-Reply-To: <3da626ed-e170-4903-933c-5fde0d2338cc@googlegroups.com> References: <3da626ed-e170-4903-933c-5fde0d2338cc@googlegroups.com> Message-ID: <39f92f9e-3a9b-2bf3-8936-20d3e2c6283a@kynesim.co.uk> On 08/06/17 00:56, CB wrote: > Hi everyone, > I am taking a python class and I'm stuck in an exercise. > > what am i doing wrong? Can anyone try to run it? Thanks so much! It helps if you describe what is going wrong. Not just us, either; "Teddy Bear Debugging", explaining to a colleague (or indeed a soft toy) why your code cannot possibly be wrong, is an excellent way of finding bugs. The number of times I've stopped in the middle of an explanation to fix the now glaringly obvious mistake... Try it and see. Here's a hint: be very aware of what your indentation means. -- Rhodri James *-* Kynesim Ltd From gvm2121 at gmail.com Thu Jun 8 07:49:02 2017 From: gvm2121 at gmail.com (Gonzalo V) Date: Thu, 8 Jun 2017 07:49:02 -0400 Subject: python certification Message-ID: hi, good day. where can i get a python certification? thanks! From mal at europython.eu Thu Jun 8 08:26:22 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Thu, 8 Jun 2017 14:26:22 +0200 Subject: EuroPython 2017: Full schedule now online Message-ID: <64d5bcbb-9dce-7522-b622-6a7857802a47@europython.eu> We are happy to announce the schedule for EuroPython 2017 in Rimini, Italy (July 9-16). The program WG has been working hard trying to fit all the sessions in the last few weeks. With over 200 sessions, over 180 speakers, one day for workshops, 5 days of talks, training, keynotes, lightning talks, posters, help desks and open spaces, followed by 2 days of sprints, EuroPython will be one of the most exciting and vibrant Python events this year: * EuroPython 2017 Schedule * https://ep2017.europython.eu/p3/schedule/ep2017/ The schedule is available in table and list format. Please note that we are still applying small changes to the slots where necessary. Many thanks to everyone who submitted proposals. EuroPython wouldn?t be possible without our speakers ! If you want to join the fun, be sure to get your tickets as soon as possible, since ticket sales usually start picking up quite a bit after we announce the schedule. Aside: If you haven?t done yet, please get your EuroPython 2017 ticket soon. We will switch to on-desk rates later in June, which will cost around 30% more than the regular rates. https://ep2017.europython.eu/en/registration/buy-tickets/ Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/872790385055211520 Thanks. From gbs.deadeye at gmail.com Thu Jun 8 08:39:53 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=c3=bcller?=) Date: Thu, 8 Jun 2017 14:39:53 +0200 Subject: Hello from a super noob! In-Reply-To: <3da626ed-e170-4903-933c-5fde0d2338cc@googlegroups.com> References: <3da626ed-e170-4903-933c-5fde0d2338cc@googlegroups.com> Message-ID: <6f8da87e-6f33-9ee8-c966-9fb1c2e12c26@gmail.com> Hello, you can refactor your code a little bit and learn more about exceptions: def get_numbers(): first = None second = None while True: try: if first is None: first = int(input('Enter your first number: ')) if second is None: second = int(input('Enter your second number: ')) except ValueError: print('You have to enter a number') continue else: return first, second Am 08.06.2017 um 01:56 schrieb CB: > Hi everyone, > I am taking a python class and I'm stuck in an exercise. > > what am i doing wrong? Can anyone try to run it? Thanks so much! > > #Description:Input validation and while loops. > > > import random > def main(): #main function need in all programs for automated testing > > > #your program goes here > > print() > > > > > print("This program will help us practice input validation and while loops.") > print("The user will be asked to enter two numbers which will both be validated. ") > print("The sum of the numbers will then be displayed in a complex print statement ") > print("and the user will be asked if they would like to run the program again." > ) > print() > print() > > while True: > FirstNumber = input ("Please enter the first number: ") > if FirstNumber.isdigit (): > FirstNumber = int(FirstNumber) > break > else: > print ("Invalid response. Please enter a whole number. " ) > > while True: > > SecondNumber = input ("Please enter the second number: " ) > if SecondNumber.isdigit(): > SecondNumber= int(SecondNumber) > > break > else: > print("Invalid response. Please enter a whole number." ) > > print() > print (str(FirstNumber) + " + " + str(SecondNumber)+ " = " + str(FirstNumber + SecondNumber)) > print() > > while True: > > ans= input('Would you like to run the program again (Y/N) : ') > if ans== 'Y' or ans== 'N': > break > > else: > print(" lnvalid response. Please answer with 'Y' or 'N' ") > > if ans== 'N': > break > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From sivas.postbox at gmail.com Thu Jun 8 11:10:07 2017 From: sivas.postbox at gmail.com (Siva Kumar S) Date: Thu, 8 Jun 2017 08:10:07 -0700 (PDT) Subject: Reg : Wikipedia 1.4 package Message-ID: <657918cc-b062-4ab4-8472-b66d515e3640@googlegroups.com> Dear members, how to install wikipedia 1.4 package in python 2.7 above without PIP. from, Sivakumar S From sivas.postbox at gmail.com Thu Jun 8 11:11:48 2017 From: sivas.postbox at gmail.com (Siva Kumar S) Date: Thu, 8 Jun 2017 08:11:48 -0700 (PDT) Subject: Reg : Wikipedia 1.4 package Message-ID: Dear members, How to install Wikipedia 1.4 package in python 2.7 above without PIP. from, Sivakumar S From israel at ravnalaska.net Thu Jun 8 12:47:59 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Thu, 8 Jun 2017 08:47:59 -0800 Subject: Psycopg2 pool clarification In-Reply-To: <87bmpzxa6t.fsf@handshake.de> References: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> <4cdc7e80393a42f209e8ca0742ace3fa@ravnalaska.net> <87k24ob856.fsf@handshake.de> <9d535a63e0013a93d741d097815c254a@ravnalaska.net> <87bmpzxa6t.fsf@handshake.de> Message-ID: <43C9C234-BA40-4C22-B6E1-3AEB70BF399B@ravnalaska.net> ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- > On Jun 7, 2017, at 10:31 PM, dieter wrote: > > israel writes: >> On 2017-06-06 22:53, dieter wrote: >> ... >> As such, using psycopg2's pool is essentially >> worthless for me (plenty of use for it, i'm sure, just not for me/my >> use case). > > Could you not simply adjust the value for the "min" parameter? > If you want at least "n" open connections, then set "min" to "n". Well, sure, if I didn't care about wasting resources (which, I guess many people don't). I could set "n" to some magic number that would always give "enough" connections, such that my application never has to open additional connections, then adjust that number every few months as usage changes. In fact, now that I know how the logic of the pool works, that's exactly what I'm doing until I am confident that my caching replacement is solid. Of course, in order to avoid having to open/close a bunch of connections during the times when it is most critical - that is, when the server is under heavy load - I have to set that number arbitrarily high. Furthermore, that means that much of the time many, if not most, of those connections would be idle. Each connection uses a certain amount of RAM on the server, not to mention using up limited connection slots, so now I've got to think about if my server is sized properly to be able to handle that load not just occasionally, but constantly - when reducing server load by reducing the frequency of connections being opened/closed was the goal in the first place. So all I've done is trade dynamic load for static load - increasing performance at the cost of resources, rather than more intelligently using the available resources. All-in-all, not the best solution, though it does work. Maybe if load was fairly constant it would make more sense though. So like I said *my* use case, which is a number of web apps with varying loads, loads that also vary from day-to-day and hour-to-hour. On the other hand, a pool that caches connections using the logic I laid out in my original post would avoid the issue. Under heavy load, it could open additional connections as needed - a performance penalty for the first few users over the min threshold, but only the first few, rather than all the users over a certain threshold ("n"). Those connections would then remain available for the duration of the load, so it doesn't need to open/close numerous connections. Then, during periods of lighter load, the unused connections can drop off, freeing up server resources for other uses. A well-written pool could even do something like see that the available connection pool is running low, and open a few more connections in the background, thus completely avoiding the connection overhead on requests while never having more than a few "extra" connections at any given time. Even if you left of the expiration logic, it would still be an improvement, because while unused connections wouldn't drop, the "n" open connections could scale up dynamically until you have "enough" connections, without having to figure out and hard-code that "magic number" of open connections. Why wouldn't I want something like that? It's not like its hard to code - took me about an hour and a half to get to a working prototype yesterday. Still need to write tests and add some polish, but it works. Perhaps, though, the common thought is just "throw more hardware at it and keep a lot of connections open at all time?" Maybe I was raised to conservatively, or the company I work for is too poor.... :-D > > -- > https://mail.python.org/mailman/listinfo/python-list From bgailer at gmail.com Thu Jun 8 13:18:22 2017 From: bgailer at gmail.com (Bob Gailer) Date: Thu, 8 Jun 2017 13:18:22 -0400 Subject: python certification In-Reply-To: References: Message-ID: On Jun 8, 2017 7:58 AM, "Gonzalo V" wrote: > > hi, > good day. > where can i get a python certification? I'm not sure there is such a thing. Try Googling. > thanks! > -- > https://mail.python.org/mailman/listinfo/python-list From walters.justin01 at gmail.com Thu Jun 8 13:51:52 2017 From: walters.justin01 at gmail.com (justin walters) Date: Thu, 8 Jun 2017 10:51:52 -0700 Subject: python certification In-Reply-To: References: Message-ID: On Thu, Jun 8, 2017 at 4:49 AM, Gonzalo V wrote: > hi, > good day. > where can i get a python certification? > > thanks! > -- > https://mail.python.org/mailman/listinfo/python-list > I don't believe there is any official certification. Nor do I believe that a Python certification would be something employers are even looking for. That said, if you really need some document that says you know Python, edX and udacity do offer completion certificates for their courses I believe. From ned at nedbatchelder.com Thu Jun 8 13:53:27 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 8 Jun 2017 10:53:27 -0700 (PDT) Subject: Reg : Wikipedia 1.4 package In-Reply-To: <657918cc-b062-4ab4-8472-b66d515e3640@googlegroups.com> References: <657918cc-b062-4ab4-8472-b66d515e3640@googlegroups.com> Message-ID: On Thursday, June 8, 2017 at 11:10:49 AM UTC-4, Siva Kumar S wrote: > Dear members, > > how to install wikipedia 1.4 package in python 2.7 above without PIP. Why don't you want to use pip? You can probably just download the .tar.gz, unpack it, and run: python setup.py install but pip will be better. --Ned. From fabiofz at gmail.com Thu Jun 8 14:15:15 2017 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Thu, 8 Jun 2017 15:15:15 -0300 Subject: PyDev 5.8.0: Code Coverage fixes, IronPython debugging Message-ID: PyDev 5.8.0 Release Highlights - *Important* PyDev now requires Java 8 and Eclipse 4.6 (Neon) onwards. - PyDev 5.2.0 is the last release supporting Eclipse 4.5 (Mars). - *Code Analysis* - Fixed issue getting existing PyLint markers. - There's now an Info and an Ignore level. - *Debugger* - The debugger now provides hooks for clients and provides ways to extend the handling of custom types. See: https://github.com/fabioz/PyDev.Debugger/tree/master/pydevd_plugins/extensions (patch by *Yuli Fiterman*). - Fixed issue where the debugger could end up removing quotes on args. *#PyDev-797* - The debugger now works with IronPython again -- although note that *IronPython* *2.7.6* and *2.7.7* have a critical bug which prevents IronPython from working in PyDev: https://github.com/IronLanguages/main/issues/1663 - *Code Coverage* - Fixed issue getting code-coverage version. *#PyDev-791* - Properly works when running with pytest (provided that pytest-cov is installed). - *Others* - Update .yaml file for google app engine project templates (patch by *JunjieW*). - Upgraded Lucene to 6.1.0 (patch by *Sopot Cela*). - Update docstring from parameters (Ctrl+1 on *def*) properly considers sphinx with types. *#PyDev-787* - Code Completion: Properly finding *__init__* from superclass in inherited classes. *#PyDev-802* - No longer showing icon to start interactive console in toolbar because Eclipse could end up creating multiple entries which were shown forever. *#PyDev-708* - Other minor bugfixes. What is PyDev? PyDev is an open-source Python IDE on top of Eclipse for Python, Jython and IronPython development. It comes with goodies such as code completion, syntax highlighting, syntax analysis, code analysis, refactor, debug, interactive console, etc. Details on PyDev: http://pydev.org Details on its development: http://pydev.blogspot.com What is LiClipse? LiClipse is a PyDev standalone with goodies such as support for Multiple cursors, theming, TextMate bundles and a number of other languages such as Django Templates, Jinja2, Kivy Language, Mako Templates, Html, Javascript, etc. It's also a commercial counterpart which helps supporting the development of PyDev. Details on LiClipse: http://www.liclipse.com/ Cheers, -- Fabio Zadrozny ------------------------------ Software Developer LiClipse http://www.liclipse.com PyDev - Python Development Environment for Eclipse http://pydev.org http://pydev.blogspot.com PyVmMonitor - Python Profiler http://www.pyvmmonitor.com/ From fred at bristle.com Thu Jun 8 16:54:01 2017 From: fred at bristle.com (Fred Stluka) Date: Thu, 8 Jun 2017 16:54:01 -0400 Subject: New to Python - Career question In-Reply-To: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> References: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> Message-ID: <6e26b459-44e2-6a69-c684-ba8bc7a49794@bristle.com> On 6/6/17 5:39 PM, [1]ptanyc at gmail.com wrote: New to Python and have been at it for about a month now. I'm doing well and like it very much. Considering a career change down the road and have been wondering... What are the job prospects for a middle age entry level programmer. Just trying to get a better understanding where I stand career wise. Appreciate all feed back. Thanks! Are you asking about job prospects based on experience with Python vs other languages? If so, here's a site that shows the relative demand for various programming languages over time. Based on its huge database of job listings. Also shows the relative supply of programmers in various languages over time. You can specify which languages or other marketable skills to show in the graph: - [2]https://www.indeed.com/jobtrends/q-python-q-ruby-q-php-q-javascript-q-java-q-perl.html Hope this helps, --Fred -------------------------------------------------------------------------- Fred Stluka -- [3]mailto:fred at bristle.com -- [4]http://bristle.com/~fred/ Bristle Software, Inc -- [5]http://bristle.com -- Glad to be of service! Open Source: Without walls and fences, we need no Windows or Gates. -------------------------------------------------------------------------- References Visible links 1. mailto:ptanyc at gmail.com 2. https://www.indeed.com/jobtrends/q-python-q-ruby-q-php-q-javascript-q-java-q-perl.html 3. mailto:fred at bristle.com 4. http://bristle.com/~fred/ 5. http://bristle.com/ From juan0christian at gmail.com Thu Jun 8 21:40:10 2017 From: juan0christian at gmail.com (Juan C.) Date: Thu, 8 Jun 2017 22:40:10 -0300 Subject: How to run self-contained Python scripts who don't need Python installation? Message-ID: I need to run some Python 3.6.0 scripts on the users' machines (W7 and W10) in an enterprise environment, but I can't install Python on those machines. I tried looking for those "py to exe", but sadly they don't support Python 3.6.0. Then I found out about "Windows x86/x86-64 embeddable zip file" that Python.org made available since 3.5.0, and I thought it would be the right thing for my case. I have some questions regarding embeddable Python: 1. How can I install custom modules using pip? 2. I'd like to create a simple BAT to run my Python script, so there would be only 2 things (a 'dist' folder with everything and a run.bat to make it clear what should be run), for example: @echo off start %~dp0dist\python-3.6.0-embed-win32\python.exe %~dp0dist\app.py The issue is that I can't parse args to app.py in this case. How would that be possible? In the example above, if Python was fully installed, I would give a file as argument to app.py, generally clicking and dragging the file into the app.py icon so that it would start. 3. Is there a better approach for my case other than embedded Python? From torriem at gmail.com Thu Jun 8 22:56:44 2017 From: torriem at gmail.com (Michael Torrie) Date: Thu, 8 Jun 2017 20:56:44 -0600 Subject: How to run self-contained Python scripts who don't need Python installation? In-Reply-To: References: Message-ID: On 06/08/2017 07:40 PM, Juan C. wrote: > 2. I'd like to create a simple BAT to run my Python script, so there > would be only 2 things (a 'dist' folder with everything and a run.bat > to make it clear what should be run), for example: Sure you can. You just have to do it like this (untested; I haven't used Windows in many years): @echo off start %~dp0dist\python-3.6.0-embed-win32\python.exe %~dp0dist\app.py %* I think %* expands to all the commandline parameters, apparently excluding %0, which would be the command name itself. From ian.g.kelly at gmail.com Thu Jun 8 23:55:30 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 8 Jun 2017 21:55:30 -0600 Subject: Psycopg2 pool clarification In-Reply-To: <43C9C234-BA40-4C22-B6E1-3AEB70BF399B@ravnalaska.net> References: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> <4cdc7e80393a42f209e8ca0742ace3fa@ravnalaska.net> <87k24ob856.fsf@handshake.de> <9d535a63e0013a93d741d097815c254a@ravnalaska.net> <87bmpzxa6t.fsf@handshake.de> <43C9C234-BA40-4C22-B6E1-3AEB70BF399B@ravnalaska.net> Message-ID: On Thu, Jun 8, 2017 at 10:47 AM, Israel Brewster wrote: >> On Jun 7, 2017, at 10:31 PM, dieter wrote: >> >> israel writes: >>> On 2017-06-06 22:53, dieter wrote: >>> ... >>> As such, using psycopg2's pool is essentially >>> worthless for me (plenty of use for it, i'm sure, just not for me/my >>> use case). >> >> Could you not simply adjust the value for the "min" parameter? >> If you want at least "n" open connections, then set "min" to "n". > > Well, sure, if I didn't care about wasting resources (which, I guess many people don't). I could set "n" to some magic number that would always give "enough" connections, such that my application never has to open additional connections, then adjust that number every few months as usage changes. In fact, now that I know how the logic of the pool works, that's exactly what I'm doing until I am confident that my caching replacement is solid. > > Of course, in order to avoid having to open/close a bunch of connections during the times when it is most critical - that is, when the server is under heavy load - I have to set that number arbitrarily high. Furthermore, that means that much of the time many, if not most, of those connections would be idle. Each connection uses a certain amount of RAM on the server, not to mention using up limited connection slots, so now I've got to think about if my server is sized properly to be able to handle that load not just occasionally, but constantly - when reducing server load by reducing the frequency of connections being opened/closed was the goal in the first place. So all I've done is trade dynamic load for static load - increasing performance at the cost of resources, rather than more intelligently using the available resources. All-in-all, not the best solution, though it does work. Maybe if load was fairly constant it would make more sense though. So like I said *my* use case, which > is a number of web apps with varying loads, loads that also vary from day-to-day and hour-to-hour. > > On the other hand, a pool that caches connections using the logic I laid out in my original post would avoid the issue. Under heavy load, it could open additional connections as needed - a performance penalty for the first few users over the min threshold, but only the first few, rather than all the users over a certain threshold ("n"). Those connections would then remain available for the duration of the load, so it doesn't need to open/close numerous connections. Then, during periods of lighter load, the unused connections can drop off, freeing up server resources for other uses. A well-written pool could even do something like see that the available connection pool is running low, and open a few more connections in the background, thus completely avoiding the connection overhead on requests while never having more than a few "extra" connections at any given time. Even if you left of the expiration logic, it would still be an improvement, because while unused connections wouldn't d > rop, the "n" open connections could scale up dynamically until you have "enough" connections, without having to figure out and hard-code that "magic number" of open connections. > > Why wouldn't I want something like that? It's not like its hard to code - took me about an hour and a half to get to a working prototype yesterday. Still need to write tests and add some polish, but it works. Perhaps, though, the common thought is just "throw more hardware at it and keep a lot of connections open at all time?" Maybe I was raised to conservatively, or the company I work for is too poor.... :-D Psycopg is first and foremost a database adapter. To quote from the psycopg2.pool module documentation, "This module offers a few pure Python classes implementing *simple* connection pooling directly in the client application" (emphasis added). The advertised list of features at http://initd.org/psycopg/features/ doesn't even mention connection pooling. In short, you're getting what you paid for. It sounds like your needs are beyond what the psycopg2.pool module provides. I suggest looking into a dedicated connection pooler like PgBouncer. You'll find that it's much more feature-rich and configurable than psycopg2.pool. It's production-ready, unlike your prototype. And since it's a proxy, it can take connections from multiple client apps and tune the pool to your overall load rather than on an app-by-app basis (and thus risk overloading the backend if multiple apps unexpectedly peak together). As for why psycopg2.pool is the way it is, maybe most users don't have your situation of serving multiple apps with loads varying on different cycles. Most are probably only serving a single app, or if serving multiple apps then they likely have common user bases with similar peak times. You can't dynamically adjust the amount of RAM in your server, so saving resources like RAM at below-peak times only matters if you're going to do something else with it. In the scenarios I described there isn't much else to do with it, so I can understand if saving RAM isn't a priority. From 4kir4.1i at gmail.com Fri Jun 9 07:26:34 2017 From: 4kir4.1i at gmail.com (Akira Li) Date: Fri, 09 Jun 2017 14:26:34 +0300 Subject: How to run self-contained Python scripts who don't need Python installation? References: Message-ID: <87zidhz9jp.fsf@gmail.com> "Juan C." writes: > I need to run some Python 3.6.0 scripts on the users' machines (W7 and > W10) in an enterprise environment, but I can't install Python on those > machines. I tried looking for those "py to exe", but sadly they don't > support Python 3.6.0. I've tried PyInstaller (development version) and it works with Python 3.6: $ py -3.6 -m pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip From https://www.reddit.com/r/Python/comments/6fpr70/how_do_i_distribute_a_py_file_and_its/ From steve+python at pearwood.info Fri Jun 9 07:32:03 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 09 Jun 2017 21:32:03 +1000 Subject: Bug or intended behavior? (Posting On Python-List Prohibited) References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <874lvutevs.fsf@elektro.pacujo.net> <680ae619-5b0e-4f54-bc80-1dd1115b4318@googlegroups.com> Message-ID: <593a8735$0$1593$c3e8da3$5496439d@news.astraweb.com> On Fri, 9 Jun 2017 02:23 pm, Lawrence D?Oliveiro wrote: > The indentation itself doesn?t provide enough grouping, cf > . Your argument is: "Now, what happens if these pieces of code get posted online somewhere, say in a discussion forum which makes it hard, or even impossible to keep the correct formatting?" "Doctor, it hurts when I stab myself in the eye with this pointy stick." "Then stop doing that." If your forum strips leading whitespace, the forum software is broken and should be avoided if possible. If not, don't blame Python for your decision to use broken tools. You might as well argue that C's "greatest mistake" is to use braces for grouping code blocks instead of BEGIN/END like Pascal, because there exists broken software that strips { and } characters from your text. Why would they do that? Simple: { and } don't exist in the original 1963 version of the ASCII standard. For many years, very few computers supported braces. That's why Pascal had *two* delimiters for comments: { comment } (* comment *) It wasn't until 1967, four years after the initial ASCII standard was published, that braces were added to the language. So any program which stripped out "unassigned" characters from your source code would remove braces. If we came across such a program, would we: - argue that C and other brace languages are to blame? - or that the *program*, not the language, is broken? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Jun 9 07:52:13 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 09 Jun 2017 21:52:13 +1000 Subject: Bug or intended behavior? (Posting On Python-List Prohibited) References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <5d0eaeff-55b1-5989-df92-6f3c2f6a33e7@gmail.com> <3c3d3d9f-9ee9-4610-a232-2d0565dcf061@googlegroups.com> <65d7eebe-e5be-40de-88c0-ec8bacb8ecbb@googlegroups.com> Message-ID: <593a8bf0$0$1620$c3e8da3$5496439d@news.astraweb.com> On Fri, 9 Jun 2017 02:13 pm, Lawrence D?Oliveiro wrote: > On Sunday, June 4, 2017 at 9:59:11 AM UTC+12, Sean DiZazzo wrote: >> Looking at operator precedence, I only see the % operator in regards to >> modulus. Nothing in regards to string formatting. > > Operators in Python have no intrinsic meaning. They are just syntactic sugar > for invoking methods with special names on the operand objects. That's wrong. The dunder methods are implementation, and are not intended to be called directly. It is a myth that, for example: > For example, > > a % b > > is just a cleaner way of writing > > a.__mod__(b) > > or, if object ?a? does not support ?__mod__?, but object ?b? has ?__rmod__?, > then it means > > b.__rmod__(a) That's wrong. First error: in CPython 3, and in CPython 2 for new style classes, the interpreter does not call a.__mod__ or b.__rmod__. Instead, it calls type(a).__mod__ or type(b).__rmod__, which is not necessarily the same thing. Second error: it is not correct that b.__rmod__ is only called if a does not support __mod__. That is a woefully incomplete description of what actually occurs. Here is an approximate pseudo-code of what actually happens: A = type(a) B = type(b) if issubclass(B, A) and hasattr(B, '__rmod__'): x = B.__rmod__(b, a) if x is NotImplemented: if hasattr(A, '__mod__'): x = A.__mod__(a, b) if x is NotImplemented: raise TypeError else: return x else: return x elif hasattr(A, '__mod__'): x = A.__mod__(a, b) if x is NotImplemented: if hasattr(B, '__rmod__'): x = B.__mod__(b, a) if x is NotImplemented: raise TypeError else: return x else: return x elif hasattr(B, '__rmod__'): x = B.__rmod__(b, a) if x is NotImplemented: assert not hasattr(A, '__mod__') raise TypeError else: return x else: raise TypeError The bottom line is, if you are calling dunder methods directly instead of operators, you're probably doing it wrong. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Jun 9 07:52:32 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 09 Jun 2017 21:52:32 +1000 Subject: Is An Element of a Sequence an Object? References: <5248e72e-5804-4699-bc09-8bf200589a00@googlegroups.com> <5933daa8$0$1564$e4fe514c@news.kpn.nl> <5935254a$0$1715$e4fe514c@news.kpn.nl> Message-ID: <593a8c02$0$1620$c3e8da3$5496439d@news.astraweb.com> On Fri, 9 Jun 2017 02:04 pm, Lawrence D?Oliveiro wrote: > I generally have very little use for ?is? in my code. I do try it out for > testing sometimes, to ensure consistency of semantic behaviour in library > bindings for example, but I don?t think I have ever written an ?is?-construct > in production code. If you are using None as a sentinel value, and testing for None with == (equality) instead of identity (is), then you have a bug waiting to strike. if obj == None: # do special behaviour that is intended to ONLY occur if obj is None Using equality there is wrong (as well as needlessly slow). The problem is that Python will call obj.__eq__ which has a chance to claim to be equal to None. If it does, then you will run the "only run for None" code for something that isn't None. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From sondeskalboussi at gmail.com Fri Jun 9 14:39:55 2017 From: sondeskalboussi at gmail.com (sondes kalboussi) Date: Fri, 9 Jun 2017 11:39:55 -0700 (PDT) Subject: In which order many functions are executed in a python code Message-ID: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> Am a bit confused I was thinking that the order of execution of functions in a code is from the first to the last function but sometimes it is the opposite, for instance, some parameters or outputs from the second function are called in the first one even thou they are not global, any hints ? From tomuxiong at gmx.com Fri Jun 9 14:46:43 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Fri, 9 Jun 2017 11:46:43 -0700 Subject: In which order many functions are executed in a python code In-Reply-To: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> References: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> Message-ID: <1ff5094c-10f4-27fe-aad5-a4277c8b746f@gmx.com> On 06/09/2017 11:39 AM, sondes kalboussi wrote: > > Am a bit confused I was thinking that the order of execution of functions in a code is from the first to the last function but sometimes it is the opposite, for instance, some parameters or outputs from the second function are called in the first one even thou they are not global, any hints ? > Maybe if you post some specific code it will be easier to see what you're confused about? Cheers, Thomas From joel.goldstick at gmail.com Fri Jun 9 14:56:16 2017 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Fri, 9 Jun 2017 14:56:16 -0400 Subject: In which order many functions are executed in a python code In-Reply-To: <1ff5094c-10f4-27fe-aad5-a4277c8b746f@gmx.com> References: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> <1ff5094c-10f4-27fe-aad5-a4277c8b746f@gmx.com> Message-ID: On Fri, Jun 9, 2017 at 2:46 PM, Thomas Nyberg wrote: > On 06/09/2017 11:39 AM, sondes kalboussi wrote: >> >> Am a bit confused I was thinking that the order of execution of functions in a code is from the first to the last function but sometimes it is the opposite, for instance, some parameters or outputs from the second function are called in the first one even thou they are not global, any hints ? >> > Maybe if you post some specific code it will be easier to see what Functions are run in the order in which they are called from the main code -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays From subhabangalore at gmail.com Fri Jun 9 15:14:26 2017 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Fri, 9 Jun 2017 12:14:26 -0700 (PDT) Subject: Time Calculation to Tag a Sentence/File (Posting On Python-List Prohibited) In-Reply-To: References: Message-ID: On Friday, June 9, 2017 at 1:18:35 PM UTC+5:30, Lawrence D?Oliveiro wrote: > On Thursday, June 8, 2017 at 9:57:40 AM UTC+12, subhaba... at gmail.com wrote: > > ... (with Python2.7 on MS-Windows 7) ... > > Why? Are you asking why not Python3? My Java based colleagues say it clashes with Java, so we try to work around Python2.x. From paul.james.barry at gmail.com Fri Jun 9 16:22:32 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Fri, 9 Jun 2017 21:22:32 +0100 Subject: Time Calculation to Tag a Sentence/File (Posting On Python-List Prohibited) In-Reply-To: References: Message-ID: This is a strange statement. Python 3 doesn't even clash with Python 2, so I can't think of how it might cause problems with Java. I've run 2 and 3 on Windows 7, Vista, and 10 without any issues. Paul. On 9 June 2017 at 20:14, wrote: > On Friday, June 9, 2017 at 1:18:35 PM UTC+5:30, Lawrence D?Oliveiro wrote: > > On Thursday, June 8, 2017 at 9:57:40 AM UTC+12, subhaba... at gmail.com > wrote: > > > ... (with Python2.7 on MS-Windows 7) ... > > > > Why? > > Are you asking why not Python3? My Java based colleagues say it clashes > with Java, so we try to work around Python2.x. > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From subhabangalore at gmail.com Fri Jun 9 17:31:07 2017 From: subhabangalore at gmail.com (subhabangalore at gmail.com) Date: Fri, 9 Jun 2017 14:31:07 -0700 (PDT) Subject: Time Calculation to Tag a Sentence/File (Posting On Python-List Prohibited) In-Reply-To: References: Message-ID: <9a9c1acd-30d4-4aa1-ba3a-eeaa15276fa1@googlegroups.com> On Saturday, June 10, 2017 at 1:53:07 AM UTC+5:30, Paul Barry wrote: > This is a strange statement. Python 3 doesn't even clash with Python 2, so > I can't think of how it might cause problems with Java. I've run 2 and 3 > on Windows 7, Vista, and 10 without any issues. > > Paul. > > On 9 June 2017 at 20:14, wrote: > > > On Friday, June 9, 2017 at 1:18:35 PM UTC+5:30, Lawrence D?Oliveiro wrote: > > > > > > > ... (with Python2.7 on MS-Windows 7) ... > > > > > > Why? > > > > Are you asking why not Python3? My Java based colleagues say it clashes > > with Java, so we try to work around Python2.x. > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > > -- > > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. Dear Sir, I believe I can take your word. I'd take your word and send to my project manager, let me check his view now. Regards, RP From python at lucidity.plus.com Fri Jun 9 18:00:37 2017 From: python at lucidity.plus.com (Erik) Date: Fri, 9 Jun 2017 23:00:37 +0100 Subject: In which order many functions are executed in a python code In-Reply-To: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> References: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> Message-ID: On 09/06/17 19:39, sondes kalboussi wrote: > Am a bit confused I was thinking that the order of execution of > functions in a code is from the first to the last function but > sometimes it is the opposite, for instance, some parameters or > outputs from the second function are called in the first one even > thou they are not global, any hints ? As a complete and utter guess, I assume you are talking about something like: result = func1(1, 2, func2(x, y)) In this case, func2() will be called before func1(). This is because func1() needs three parameters and one of those parameters is the return value of func2(). Python can not know the return value of func2() without calling it. Therefore, to be able to call func1() and give it its three parameters, it must first call func2() to find out what that third parameter value is. It's equivalent to: func2result = func2(x, y) result = func1(1, 2, func2result) If that is _not_ what you are talking about, then like Thomas says - you need to paste some code and explain what you are confused by. E. From tjreedy at udel.edu Fri Jun 9 19:18:52 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 9 Jun 2017 19:18:52 -0400 Subject: In which order many functions are executed in a python code In-Reply-To: References: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> Message-ID: On 6/9/2017 6:00 PM, Erik wrote: > On 09/06/17 19:39, sondes kalboussi wrote: >> Am a bit confused I was thinking that the order of execution of >> functions in a code is from the first to the last function but >> sometimes it is the opposite, for instance, some parameters or >> outputs from the second function are called in the first one even >> thou they are not global, any hints ? > > As a complete and utter guess, I assume you are talking about something > like: > > result = func1(1, 2, func2(x, y)) On the right side of '=', Python evaluates expressions, to the extent possible, left to right. The result of each evaluation is an object. In the above, the order is func1, 1, 2, func2, x, y, _tem = func2(x, y), func1(1, 2, _tem). Note that function expressions can be more complicated than just a name, as in func_array[selector]. > In this case, func2() will be called before func1(). This is because > func1() needs three parameters and one of those parameters is the return > value of func2(). Given r = f[a](1, g[b](c)), your rule above does not determine whether f[a] or g[b] is determined first. In at least some C implementations, g[b] would be. Since the call g[b](c) could affect the bindings of f, a, and the result of f[a], the rule that f[a] is calculated before g, b, c, g[b], and g[b](c) may make a real difference. > Python can not know the return value of func2() without calling it. > Therefore, to be able to call func1() and give it its three parameters, > it must first call func2() to find out what that third parameter value is. > > It's equivalent to: > > func2result = func2(x, y) > result = func1(1, 2, func2result) Since the func2 call could have the side effect of rebinding 'func1', this is not exactly equivalent. > If that is _not_ what you are talking about, then like Thomas says - you > need to paste some code and explain what you are confused by. Indeed. -- Terry Jan Reedy From python at lucidity.plus.com Fri Jun 9 20:14:40 2017 From: python at lucidity.plus.com (Erik) Date: Sat, 10 Jun 2017 01:14:40 +0100 Subject: In which order many functions are executed in a python code In-Reply-To: References: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> Message-ID: <4104b2b0-254b-bba5-42f1-b6bde46d5be0@lucidity.plus.com> On 10/06/17 00:18, Terry Reedy wrote: > On 6/9/2017 6:00 PM, Erik wrote: >> On 09/06/17 19:39, sondes kalboussi wrote: >>> Am a bit confused I was thinking that the order of execution of >>> functions in a code is from the first to the last function but >>> sometimes it is the opposite, for instance, some parameters or >>> outputs from the second function are called in the first one even >>> thou they are not global, any hints ? >> >> As a complete and utter guess, I assume you are talking about >> something like: >> >> result = func1(1, 2, func2(x, y)) > > On the right side of '=', Python evaluates expressions, to the extent > possible, left to right. The result of each evaluation is an object. In > the above, the order is func1, 1, 2, func2, x, y, _tem = func2(x, y), > func1(1, 2, _tem). Note that function expressions can be more > complicated than just a name, as in func_array[selector]. Terry, how does this help the OP who is obviously a learner, understand their problem? >> In this case, func2() will be called before func1(). This is because >> func1() needs three parameters and one of those parameters is the >> return value of func2(). > > Given r = f[a](1, g[b](c)), your rule above does not determine whether > f[a] or g[b] is determined first. In at least some C implementations, > g[b] would be. Since the call g[b](c) could affect the bindings of f, > a, and the result of f[a], the rule that f[a] is calculated before g, b, > c, g[b], and g[b](c) may make a real difference. Terry, how does this help the OP who is obviously a learner, understand their problem? >> Python can not know the return value of func2() without calling it. >> Therefore, to be able to call func1() and give it its three >> parameters, it must first call func2() to find out what that third >> parameter value is. >> >> It's equivalent to: >> >> func2result = func2(x, y) >> result = func1(1, 2, func2result) > > Since the func2 call could have the side effect of rebinding 'func1', > this is not exactly equivalent. Whilst this is strictly correct, Terry, how does this help the OP who is obviously a learner, understand their problem? I was trying to help someone by writing in terms I thought would help them to understand the question they posed (if I was correct in my assumption of what that question actually meant). Why have you just jumped on everything I've said with responses that are really not what someone posting to -list for the first time might even understand? If you want to prove something to me, then send something to me. I really don't understand why you would respond in this way. E. From FredFishbin at hotmail.com Fri Jun 9 22:04:08 2017 From: FredFishbin at hotmail.com (Fred Fishbin) Date: Fri, 09 Jun 2017 21:04:08 -0500 Subject: Access flles on a Android device from Windows PC References: Message-ID: Um, no. I *now* understand that USB Mass drive is no longer supported, but it was a legit question. I'd still love if someone could post a code sample, or point me in the direction to a code snippet that would show me how to access the phone. I know it can be done, its just not simple anymore. Freddie wxjmfauth at gmail.com wrote: Le mardi 6 juin 2017 21:37:24 UTC+2, Fred Fishbin a ??crit??: >> Hi >> >> I want to write little program that my friend can run - he'll plug a USB >>drive >> into his Windows 7 PC, plug his phone in a USB port on same PC, then run my >> program and it'll xfer some audiobook files over for him. >> >> I plugged the USB drive in and it became "G:\", but the phone plugged in and >> became just "SAMSUNG-SM-G930V", no drive designation, just a Portable Media >> Device. Windows Explore can go there so the files are accessible, the phone >> isn't looking him out, but what do I use for a path? I tried several >>different >> listdir()'s nothing worked. >> >> Thoughts? >> >> thanks, Freddie > >I hope your mounted external device is full of file names >with plenty of Emojis... >It will be very funny. From niteesh.k80 at gmail.com Fri Jun 9 23:00:52 2017 From: niteesh.k80 at gmail.com (niteesh.k80 at gmail.com) Date: Fri, 9 Jun 2017 20:00:52 -0700 (PDT) Subject: Passing arguments to & executing, a python script on a remote machine from a python script on local machine (using ssh ?) In-Reply-To: <10db83e4-80eb-4265-b11c-062269fea1b0@googlegroups.com> References: <10db83e4-80eb-4265-b11c-062269fea1b0@googlegroups.com> Message-ID: <566a0062-5ba9-4194-a499-f6447c019068@googlegroups.com> hey did you find the answer for this From FredFishbin at hotmail.com Fri Jun 9 23:52:36 2017 From: FredFishbin at hotmail.com (Fred Fishbin) Date: Fri, 09 Jun 2017 22:52:36 -0500 Subject: Access flles on a Android device from Windows PC (Posting On Python-List Prohibited) References: Message-ID: Lawrence_D?Oliveiro wrote: >On Thursday, June 8, 2017 at 3:41:35 PM UTC+12, Fred Fishbin wrote: >> Yup, that seems to be the deal, and there doesn't seem tpo be a really >>simple >> way to deal with this. > >The preferred way to do file transfers to/from Android devices over USB seems >to be via MTP nowadays . Excellent! Exactly what I was looking for! thank you, Freddie From FredFishbin at hotmail.com Fri Jun 9 23:55:57 2017 From: FredFishbin at hotmail.com (Fred Fishbin) Date: Fri, 09 Jun 2017 22:55:57 -0500 Subject: Access flles on a Android device from Windows PC (Posting On Python-List Prohibited) References: Message-ID: Lawrence_D?Oliveiro wrote: >On Thursday, June 8, 2017 at 3:41:35 PM UTC+12, Fred Fishbin wrote: >> Yup, that seems to be the deal, and there doesn't seem tpo be a really >>simple >> way to deal with this. > >The preferred way to do file transfers to/from Android devices over USB seems >to be via MTP nowadays . Excellent! Exactly what I was looking for! thank you, Freddie From tjreedy at udel.edu Fri Jun 9 23:57:45 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 9 Jun 2017 23:57:45 -0400 Subject: In which order many functions are executed in a python code In-Reply-To: <4104b2b0-254b-bba5-42f1-b6bde46d5be0@lucidity.plus.com> References: <8fd8fdce-1013-416b-867f-21b4a9a2385c@googlegroups.com> <4104b2b0-254b-bba5-42f1-b6bde46d5be0@lucidity.plus.com> Message-ID: On 6/9/2017 8:14 PM, Erik wrote: > On 10/06/17 00:18, Terry Reedy wrote: >> On 6/9/2017 6:00 PM, Erik wrote: >>> On 09/06/17 19:39, sondes kalboussi wrote: >>>> Am a bit confused I was thinking that the order of execution of >>>> functions in a code is from the first to the last function but >>>> sometimes it is the opposite, for instance, some parameters or >>>> outputs from the second function are called in the first one even >>>> thou they are not global, any hints ? >>> >>> As a complete and utter guess, I assume you are talking about >>> something like: >>> >>> result = func1(1, 2, func2(x, y)) >> On the right side of '=', Python evaluates expressions, to the extent >> possible, left to right. The result of each evaluation is an object. >> In the above, the order is func1, 1, 2, func2, x, y, _tem = func2(x, >> y), func1(1, 2, _tem). Note that function expressions can be more >> complicated than just a name, as in func_array[selector]. > > Terry, how does this help the OP who is obviously a learner, understand > their problem? Which is unclear in the absence of any example. Erik, calm down. I did not jump on what you said, I added detail. python-list is a public forum and answers are for any one who reads the thread, either now or in the future. That is why answers are posted publicly, not privately. (I happen to more often answer questions on StackOverflow, where questions and answers are more explicitly intended to constitute a growing database of programming knowledge, and where they are much more easily searched.) That said, Python's uniform left-to-right rule is a simplifying rule which I found very helpful when *I* was a beginner. It was a relief after my previous experience with a language (C) where so many behaviors are 'implementation defined' or worse, 'undefined'. So I thought it might be helpful to this beginner and I expect it will be to someone. Knowing order of evaluation is crucial to understanding code where most anything might have a side-effect. For example, what should be the order of output of this code? def echo(x): print(x) return x d = {echo(1): echo(2), echo(3): echo(4)} In 3.5+, it is 1,2,3,4, as the rule says. In early 3.x, 2.7, and some undetermined earlier versions, it was 2,1,4,3*. When the bug was discovered, there was a thought that we should just document the exception, because fixing the bug could break code. The decision was to leave maintenance versions, including 2.x, alone and fix the bug in the next py 3 version. * If one views dict displays as abbreviating a series of assignments, such as d[echo(1)] = echo(2); d[echo(3)] = echo(4), the reversal makes some sense. -- Terry Jan Reedy From no.email at nospam.invalid Sat Jun 10 03:51:02 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 10 Jun 2017 00:51:02 -0700 Subject: New to Python - Career question References: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> Message-ID: <87o9twe0wp.fsf@nightsong.com> Larry Martell writes: > I can tell they think I am old and they dismiss me right away. http://oldgeekjobs.com ? From hamp.cpas at gmail.com Sat Jun 10 05:33:19 2017 From: hamp.cpas at gmail.com (Mohammad Ghasemi) Date: Sat, 10 Jun 2017 02:33:19 -0700 (PDT) Subject: httplib HTTP class for python3? Message-ID: Hi. I want to convert a python 2 code to python 3 code and I have a problem with that. I've asked my question on stackoverflow and never received an answer. Where can I get an answer? https://stackoverflow.com/questions/44411439/httplib-http-class-for-python3 From tjol at tjol.eu Sat Jun 10 06:52:51 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 10 Jun 2017 12:52:51 +0200 Subject: httplib HTTP class for python3? In-Reply-To: References: Message-ID: <593bd016$0$1781$e4fe514c@news.kpn.nl> On 10/06/17 11:33, Mohammad Ghasemi wrote: > Hi. > I want to convert a python 2 code to python 3 code and I have a problem with that. > I've asked my question on stackoverflow and never received an answer. > Where can I get an answer? > > https://stackoverflow.com/questions/44411439/httplib-http-class-for-python3 > You asked this only two days ago. "never received an answer" is a strong statement considering. As you obviously know, httplib as has renamed http.client. > [...] from http.client import HTTP will not work. According to the documentation , the HTTP class is ancient and "should not be used in new code". It makes sense that it would have been removed in Python 3.0 - looks like you should just use HTTPConnection. More generally: If you say something does not work, please provide a complete minimal working example with an error message. Otherwise there is no way anybody can know what it going wrong. You provided example code (great!) but it's not complete: you are using things without importing them, and it doesn't look like your code would actually do much of anything if the relevant modules *were* imported. -- Thomas From hamp.cpas at gmail.com Sat Jun 10 08:09:25 2017 From: hamp.cpas at gmail.com (Mohammad Ghasemi) Date: Sat, 10 Jun 2017 05:09:25 -0700 (PDT) Subject: httplib HTTP class for python3? In-Reply-To: <593bd016$0$1781$e4fe514c@news.kpn.nl> References: <593bd016$0$1781$e4fe514c@news.kpn.nl> Message-ID: <9662816a-15de-494b-beec-51b898c79ed9@googlegroups.com> On Saturday, June 10, 2017 at 3:23:03 PM UTC+4:30, Thomas Jollans wrote: > On 10/06/17 11:33, Mohammad Ghasemi wrote: > > Hi. > > I want to convert a python 2 code to python 3 code and I have a problem with that. > > I've asked my question on stackoverflow and never received an answer. > > Where can I get an answer? > > > > https://stackoverflow.com/questions/44411439/httplib-http-class-for-python3 > > > > You asked this only two days ago. "never received an answer" is a strong > statement considering. > > > > As you obviously know, httplib as has renamed http.client. > > > [...] from http.client import HTTP will not work. > > According to the documentation > , the HTTP class is > ancient and "should not be used in new code". It makes sense that it > would have been removed in Python 3.0 - looks like you should just use > HTTPConnection. > > More generally: > > If you say something does not work, please provide a complete minimal > working example with an error message. Otherwise there is no way anybody > can know what it going wrong. > > You provided example code (great!) but it's not complete: you are using > things without importing them, and it doesn't look like your code would > actually do much of anything if the relevant modules *were* imported. > > > -- Thomas As you said, HTTP Class should no longer be used. You're right. >From now on, I'll try to ask my questions more readable and understandable. Thank you, Thomas. From steve+python at pearwood.info Sat Jun 10 08:37:19 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 10 Jun 2017 22:37:19 +1000 Subject: Python Language Summit: Keeping Python competitive Message-ID: <593be801$0$1618$c3e8da3$5496439d@news.astraweb.com> Python 3.7 (unstable) is now as fast as Python 2.7, but some of the core developers are hoping to improve matters, aiming for a 2 x speed improvement to keep Python competitive with other languages. https://lwn.net/Articles/723949/ -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Jun 10 08:54:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 10 Jun 2017 22:54:38 +1000 Subject: Progress on the Gilectomy Message-ID: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> Larry Hastings is working on removing the GIL from CPython: https://lwn.net/Articles/723949/ For those who don't know the background: - The GIL (Global Interpreter Lock) is used to ensure that only one piece of code can update references to an object at a time. - The downside of the GIL is that CPython cannot take advantage of multiple CPU cores effectively. Hence multi-threaded code is not as fast as it could be. - Past attempts to remove the GIL caused unacceptable slow-downs for single-threaded programs and code run on single-core CPUs. - And also failed to show the expected performance gains for multi-threaded programs on multi-core CPUs. (There was some gain, but not much.) Thanks Larry for your experiments on this! -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From irmen.NOSPAM at xs4all.nl Sat Jun 10 12:21:00 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Sat, 10 Jun 2017 18:21:00 +0200 Subject: Progress on the Gilectomy In-Reply-To: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: <593c1c6e$0$820$e4fe514c@news.xs4all.nl> On 10-6-2017 14:54, Steve D'Aprano wrote: > Larry Hastings is working on removing the GIL from CPython: > > https://lwn.net/Articles/723949/ Here is Larry's "How's it going" presentation from Pycon 2017 on this subject https://www.youtube.com/watch?v=pLqv11ScGsQ -irmen From israel at ravnalaska.net Sat Jun 10 12:36:55 2017 From: israel at ravnalaska.net (israel) Date: Sat, 10 Jun 2017 08:36:55 -0800 Subject: Psycopg2 pool clarification In-Reply-To: References: <6CB8E3C1-97CF-4EA9-8557-4E405B44ACDE@ravnalaska.net> <4cdc7e80393a42f209e8ca0742ace3fa@ravnalaska.net> <87k24ob856.fsf@handshake.de> <9d535a63e0013a93d741d097815c254a@ravnalaska.net> <87bmpzxa6t.fsf@handshake.de> <43C9C234-BA40-4C22-B6E1-3AEB70BF399B@ravnalaska.net> Message-ID: <399ce5b1aaf274c0e683d2314e87a9b3@ravnalaska.net> On 2017-06-08 19:55, Ian Kelly wrote: > On Thu, Jun 8, 2017 at 10:47 AM, Israel Brewster > wrote: >>> On Jun 7, 2017, at 10:31 PM, dieter wrote: >>> >>> israel writes: >>>> On 2017-06-06 22:53, dieter wrote: >>>> ... >>>> As such, using psycopg2's pool is essentially >>>> worthless for me (plenty of use for it, i'm sure, just not for me/my >>>> use case). >>> >>> Could you not simply adjust the value for the "min" parameter? >>> If you want at least "n" open connections, then set "min" to "n". >> >> Well, sure, if I didn't care about wasting resources (which, I guess >> many people don't). I could set "n" to some magic number that would >> always give "enough" connections, such that my application never has >> to open additional connections, then adjust that number every few >> months as usage changes. In fact, now that I know how the logic of the >> pool works, that's exactly what I'm doing until I am confident that my >> caching replacement is solid. >> >> Of course, in order to avoid having to open/close a bunch of >> connections during the times when it is most critical - that is, when >> the server is under heavy load - I have to set that number arbitrarily >> high. Furthermore, that means that much of the time many, if not most, >> of those connections would be idle. Each connection uses a certain >> amount of RAM on the server, not to mention using up limited >> connection slots, so now I've got to think about if my server is sized >> properly to be able to handle that load not just occasionally, but >> constantly - when reducing server load by reducing the frequency of >> connections being opened/closed was the goal in the first place. So >> all I've done is trade dynamic load for static load - increasing >> performance at the cost of resources, rather than more intelligently >> using the available resources. All-in-all, not the best solution, >> though it does work. Maybe if load was fairly constant it would make >> more sense though. So like I said *my* use c > ase, whi > ch >> is a number of web apps with varying loads, loads that also vary >> from day-to-day and hour-to-hour. >> >> On the other hand, a pool that caches connections using the logic I >> laid out in my original post would avoid the issue. Under heavy load, >> it could open additional connections as needed - a performance penalty >> for the first few users over the min threshold, but only the first >> few, rather than all the users over a certain threshold ("n"). Those >> connections would then remain available for the duration of the load, >> so it doesn't need to open/close numerous connections. Then, during >> periods of lighter load, the unused connections can drop off, freeing >> up server resources for other uses. A well-written pool could even do >> something like see that the available connection pool is running low, >> and open a few more connections in the background, thus completely >> avoiding the connection overhead on requests while never having more >> than a few "extra" connections at any given time. Even if you left of >> the expiration logic, it would still be an improvement, because while >> unused connections > wouldn't > d >> rop, the "n" open connections could scale up dynamically until you >> have "enough" connections, without having to figure out and hard-code >> that "magic number" of open connections. >> >> Why wouldn't I want something like that? It's not like its hard to >> code - took me about an hour and a half to get to a working prototype >> yesterday. Still need to write tests and add some polish, but it >> works. Perhaps, though, the common thought is just "throw more >> hardware at it and keep a lot of connections open at all time?" Maybe >> I was raised to conservatively, or the company I work for is too >> poor.... :-D > > Psycopg is first and foremost a database adapter. To quote from the > psycopg2.pool module documentation, "This module offers a few pure > Python classes implementing *simple* connection pooling directly in > the client application" (emphasis added). The advertised list of > features at http://initd.org/psycopg/features/ doesn't even mention > connection pooling. In short, you're getting what you paid for. > > It sounds like your needs are beyond what the psycopg2.pool module > provides. Quite possible. Thus the reason I was looking for clarification on how the module was intended to work - if it doesn't work in the way that I want it to, I need to look elsewhere for a solution. My main reason for posting this thread was that I was expecting it to work one way, but testing showed it working another way, so I was trying to find out if that was intentional or user error. Apparently it's intentional, so there we go - in it's current form at least, my needs are beyond what the psycopg2 pool provides. Fair enough. > I suggest looking into a dedicated connection pooler like > PgBouncer. You'll find that it's much more feature-rich and > configurable than psycopg2.pool. It's production-ready, unlike your > prototype. And since it's a proxy, it can take connections from > multiple client apps and tune the pool to your overall load rather > than on an app-by-app basis (and thus risk overloading the backend if > multiple apps unexpectedly peak together). Very true, and I've looked into that (as well as the related, but more basic, PgPool product), but it seems to me that any external proxy product like these would defeat *my* purpose for using a pool in the first place: avoiding the overhead of making/breaking many connections quickly. That is, all you have really done is gone from connecting to Postgres to connecting to PgBouncer. You are still making and breaking just as many connections. Unless connecting to PgBouncer is significantly cheaper than connecting to Postgres? This may well be the case, but I haven't yet seen anything to support that. Haven't seen anything to refute that either, however :) Of course, there may be many other features provided by such tools that would make them worthwhile, even for my use case. However, my primary goal in using a pool was avoiding the connection overhead with each request, so if a tool doesn't do that, then it isn't the right tool for me :) > > As for why psycopg2.pool is the way it is, maybe most users don't have > your situation of serving multiple apps with loads varying on > different cycles. Most are probably only serving a single app, or if > serving multiple apps then they likely have common user bases with > similar peak times. You can't dynamically adjust the amount of RAM in > your server, so saving resources like RAM at below-peak times only > matters if you're going to do something else with it. In the scenarios > I described there isn't much else to do with it, so I can understand > if saving RAM isn't a priority. True, but you would still have to deal with the minconn "magic number", unless you just adjusted it so high from the start that even if your load/use grows over time you never have to mess with it. Even in the single app use case, where you don't care about RAM usage (since there is nothing else trying to use the RAM) in order to get maximum benefit from a pool you'd have to keep an eye on your usage and make sure it never (or rarely) exceeds whatever arbitrary value you have set for minconn. Not a big deal, especially if you tune it high to begin with, but it is one more thing. Honestly, some of that is just personal issues. I have problems with code that is inefficient by design (even if there is nothing to be gained from efficiency), or that makes assumptions about things when it could be dynamic. I have often coded in such a way that a given value can be adjusted dynamically, even when the people giving me the specs say it will never change. More than once that has enabled me to respond to a "feature request" or change order by saying "it already does that". On the other hand, I am probably a poster child for "premature optimization", and often have to stop myself from optimizing code just because I can, when in reality it is not worth my time. By the same token, the idea of wasting resources - even when, as you state, there is nothing else to do with them - just rubs me the wrong way. As such, I readily acknowledge that some of my requests/statements stem from my own personal desires, and not from any actual lack/need in the product. From storchaka at gmail.com Sun Jun 11 01:11:34 2017 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sun, 11 Jun 2017 08:11:34 +0300 Subject: Progress on the Gilectomy In-Reply-To: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: 10.06.17 15:54, Steve D'Aprano ????: > Larry Hastings is working on removing the GIL from CPython: > > https://lwn.net/Articles/723949/ > > > For those who don't know the background: > > - The GIL (Global Interpreter Lock) is used to ensure that only one piece of > code can update references to an object at a time. > > - The downside of the GIL is that CPython cannot take advantage of multiple CPU > cores effectively. Hence multi-threaded code is not as fast as it could be. > > - Past attempts to remove the GIL caused unacceptable slow-downs for > single-threaded programs and code run on single-core CPUs. > > - And also failed to show the expected performance gains for multi-threaded > programs on multi-core CPUs. (There was some gain, but not much.) > > > Thanks Larry for your experiments on this! And also GIL is used for guaranteeing atomicity of many operations and consistencity of internal structures without using additional locks. Many parts of the core and the stdlib would just not work correctly in multithread environment without GIL. From stefan_ml at behnel.de Sun Jun 11 02:21:25 2017 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 11 Jun 2017 08:21:25 +0200 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: Serhiy Storchaka schrieb am 11.06.2017 um 07:11: > 10.06.17 15:54, Steve D'Aprano ????: >> Larry Hastings is working on removing the GIL from CPython: >> >> https://lwn.net/Articles/723949/ >> >> For those who don't know the background: >> >> - The GIL (Global Interpreter Lock) is used to ensure that only one piece of >> code can update references to an object at a time. >> >> - The downside of the GIL is that CPython cannot take advantage of >> multiple CPU >> cores effectively. Hence multi-threaded code is not as fast as it could be. >> >> - Past attempts to remove the GIL caused unacceptable slow-downs for >> single-threaded programs and code run on single-core CPUs. >> >> - And also failed to show the expected performance gains for multi-threaded >> programs on multi-core CPUs. (There was some gain, but not much.) >> >> >> Thanks Larry for your experiments on this! > > And also GIL is used for guaranteeing atomicity of many operations and > consistencity of internal structures without using additional locks. Many > parts of the core and the stdlib would just not work correctly in > multithread environment without GIL. And the same applies to external extension modules. The GIL is really handy when it comes to reasoning about safety and correctness of algorithms under the threat of thread concurrency. Especially in native code, where the result of an unanticipated race condition is usually a crash rather than an exception. Stefan From steve+python at pearwood.info Sun Jun 11 02:27:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 11 Jun 2017 16:27:56 +1000 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: <593ce2ee$0$1617$c3e8da3$5496439d@news.astraweb.com> On Sun, 11 Jun 2017 04:21 pm, Stefan Behnel wrote: > Serhiy Storchaka schrieb am 11.06.2017 um 07:11: >> And also GIL is used for guaranteeing atomicity of many operations and >> consistencity of internal structures without using additional locks. Many >> parts of the core and the stdlib would just not work correctly in >> multithread environment without GIL. > > And the same applies to external extension modules. The GIL is really handy > when it comes to reasoning about safety and correctness of algorithms under > the threat of thread concurrency. Especially in native code, where the > result of an unanticipated race condition is usually a crash rather than an > exception. Thank you Stefan and Serhiy! I'm tired of people complaining about the GIL as a "mistake" without acknowledging that it exists for a reason. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jfong at ms4.hinet.net Sun Jun 11 22:06:22 2017 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sun, 11 Jun 2017 19:06:22 -0700 (PDT) Subject: Strang thing in tkinter, and pdb too? Message-ID: <47eda4fc-02b8-4555-b8b2-b44ea8f4eebb@googlegroups.com> I had donwload wdiget-tour-py3.tar.gz examples from this site: http://www.hullsvle.ch/moodle/mod/resource/view.php?id=6697 and run one of its scripts canvasruler.py, I get stange result. First, when I run it in the cmd box, although I get a message in the box, but everything else is fine. The GUI works correctly. D:\Temp\widget-tour-py3>python canvasruler.py can't invoke "event" command: application has been destroyed while executing "event generate $w <>" (procedure "ttk::ThemeChanged" line 6) invoked from within "ttk::ThemeChanged" Second, I try to find out where is this message comes from. I insert a statement "import pdb; pdb.set_trace()" at the source line 33: 33 import pdb; pdb.set_trace() 34 # 35 # Note : to execute the Tk command, I need to create a Toplevel window 36 # I guess I could use the toplevel create in widget.py, buth then this 37 # module would not run stand-alone 38 # 39 temp = Tk() 40 STIPPLE_BITMAP = temp.tk.eval( 41 'image create bitmap @%s' % STIPPLE_BITMAP_FILE ) 42 ## Now I can delete the spurious window 43 temp.destroy() 44 45 46 TAB_NORMAL_STYLE = {'fill':'black', 'stipple':''} 47 TAB_INRANGE_STYLE = {'fill':'green', 'stipple':''} 48 TAB_OUTOFRANGE_STYLE = {'fill':'red', 'stipple':STIPPLE_BITMAP} 49 50 51 class RulerCanvas (Canvas): and run this script under interpreter, it stops at the point correctly and I can step through to reach the line 51 without seeing that message appear. D:\Temp\widget-tour-py3>python Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import canvasruler > d:\work\python\widget-tour-py3\canvasruler.py(39)() -> temp = Tk() (Pdb) next > d:\work\python\widget-tour-py3\canvasruler.py(40)() -> STIPPLE_BITMAP = temp.tk.eval( (Pdb) until 51 > d:\work\python\widget-tour-py3\canvasruler.py(51)() -> class RulerCanvas (Canvas): (Pdb) Third, I moved the set_trace() to line 50 and run again. This time I saw that message appears in the cmd box. D:\Temp\widget-tour-py3>python Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import canvasruler > d:\work\python\widget-tour-py3\canvasruler.py(51)() -> class RulerCanvas (Canvas): (Pdb) can't invoke "event" command: application has been destroyed while executing "event generate $w <>" (procedure "ttk::ThemeChanged" line 6) invoked from within "ttk::ThemeChanged" (Pdb) list 46 TAB_NORMAL_STYLE = {'fill':'black', 'stipple':''} 47 TAB_INRANGE_STYLE = {'fill':'green', 'stipple':''} 48 TAB_OUTOFRANGE_STYLE = {'fill':'red', 'stipple':STIPPLE_BITMAP} 49 50 import pdb; pdb.set_trace() 51 -> class RulerCanvas (Canvas): 52 53 def __init__(self, master ): 54 Canvas.__init__(self, master, width=200+RULER_LENGTH, height=100 ) 55 self.draw_ruler() 56 self.draw_well() (Pdb) My problem is why the difference? and how to find out where that message comes from? PS. I also run this test under a Win8.1/64bit machine, get the same result. PSS. I had scan files in the whole example directory and found there is no any file using ttk module. Best Regards, Jach Fong From llanitedave at birdandflower.com Sun Jun 11 22:16:24 2017 From: llanitedave at birdandflower.com (llanitedave) Date: Sun, 11 Jun 2017 19:16:24 -0700 (PDT) Subject: asyncio Behaviour Difference 3.6 vs 3.5 (Posting On Python-List Prohibited) In-Reply-To: <7484adf1-f466-40a5-800b-5655d67d46c7@googlegroups.com> References: <7484adf1-f466-40a5-800b-5655d67d46c7@googlegroups.com> Message-ID: On Sunday, June 11, 2017 at 2:42:41 AM UTC-7, Lawrence D?Oliveiro wrote: > I tried the following very simple script under both versions 3.5.3 and 3.6.1 of Python: > > import sys > import asyncio > > loop = asyncio.get_event_loop() > > async def idle() : > while True : > await asyncio.sleep(1) > #end while > #end idle > > loop.create_task(idle()) > loop.run_forever() > > After it starts running, I hit CTRL/C. Under both versions, I get the usual KeyboardInterrupt exception and traceback. However, under 3.5, I also get this: > > Task was destroyed but it is pending! > task: wait_for=> > > OK, as near as I can tell from the asyncio docs , it makes sense for this message to appear. Yet it does not come up with 3.6. Was it deemed to be too noisy, and too much trouble to silence, so it was quietly dropped? > > Yet according to the 3.6 release notes , the changes to asyncio have been backported to 3.5. So should there be a difference in behaviour between the two? > > I?m running Debian Unstable, so feel free to tell me 3.5.3 is obsolete and there is some later 3.5.x version. ;) My message in 3.5.3 is somewhat different, running Kubuntu 17.04: >>> import sys >>> import asyncio >>> loop = asyncio.get_event_loop() >>> async def idle(): ... while True: ... await asyncio.sleep(1) ... #end while ... #end idle ... >>> loop.create_task(idle()) :1>> >>> loop.run_forever() ^CTraceback (most recent call last): File "", line 1, in File "/usr/lib/python3.5/asyncio/base_events.py", line 421, in run_forever self._run_once() File "/usr/lib/python3.5/asyncio/base_events.py", line 1388, in _run_once event_list = self._selector.select(timeout) File "/usr/lib/python3.5/selectors.py", line 445, in select fd_event_list = self._epoll.poll(timeout, max_ev) KeyboardInterrupt As yu can see, I get the "Task pending coro= References: <47eda4fc-02b8-4555-b8b2-b44ea8f4eebb@googlegroups.com> Message-ID: On 6/11/2017 10:06 PM, jfong at ms4.hinet.net wrote: > I had donwload wdiget-tour-py3.tar.gz examples from this site: > http://www.hullsvle.ch/moodle/mod/resource/view.php?id=6697 > and run one of its scripts canvasruler.py, I get stange result. > > First, when I run it in the cmd box, although I get a message in the box, > but everything else is fine. The GUI works correctly. > > D:\Temp\widget-tour-py3>python canvasruler.py > can't invoke "event" command: application has been destroyed > while executing > "event generate $w <>" > (procedure "ttk::ThemeChanged" line 6) > invoked from within > "ttk::ThemeChanged" I have seen this too. It is not specific to any python/tkinter code, but is specific to exiting with some things destroyed and something not. This must be coming from tcl or tk, not python or tkinter. > Second, I try to find out where is this message comes from. I insert > a statement "import pdb; pdb.set_trace()" at the source line 33: > > 33 import pdb; pdb.set_trace() > 34 # > 35 # Note : to execute the Tk command, I need to create a Toplevel window > 36 # I guess I could use the toplevel create in widget.py, buth then this > 37 # module would not run stand-alone > 38 # > 39 temp = Tk() > 40 STIPPLE_BITMAP = temp.tk.eval( > 41 'image create bitmap @%s' % STIPPLE_BITMAP_FILE ) > 42 ## Now I can delete the spurious window > 43 temp.destroy() > 44 > 45 > 46 TAB_NORMAL_STYLE = {'fill':'black', 'stipple':''} > 47 TAB_INRANGE_STYLE = {'fill':'green', 'stipple':''} > 48 TAB_OUTOFRANGE_STYLE = {'fill':'red', 'stipple':STIPPLE_BITMAP} > 49 > 50 > 51 class RulerCanvas (Canvas): > > and run this script under interpreter, it stops at the point correctly and > I can step through to reach the line 51 without seeing that message appear. > > D:\Temp\widget-tour-py3>python > Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import canvasruler > > d:\work\python\widget-tour-py3\canvasruler.py(39)() > -> temp = Tk() > (Pdb) next > > d:\work\python\widget-tour-py3\canvasruler.py(40)() > -> STIPPLE_BITMAP = temp.tk.eval( > (Pdb) until 51 > > d:\work\python\widget-tour-py3\canvasruler.py(51)() > -> class RulerCanvas (Canvas): > (Pdb) > > Third, I moved the set_trace() to line 50 and run again. This time I saw that message appears in the cmd box. > > D:\Temp\widget-tour-py3>python > Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import canvasruler > > d:\work\python\widget-tour-py3\canvasruler.py(51)() > -> class RulerCanvas (Canvas): > (Pdb) can't invoke "event" command: application has been destroyed > while executing > "event generate $w <>" > (procedure "ttk::ThemeChanged" line 6) > invoked from within > "ttk::ThemeChanged" > > (Pdb) list > 46 TAB_NORMAL_STYLE = {'fill':'black', 'stipple':''} > 47 TAB_INRANGE_STYLE = {'fill':'green', 'stipple':''} > 48 TAB_OUTOFRANGE_STYLE = {'fill':'red', 'stipple':STIPPLE_BITMAP} > 49 > 50 import pdb; pdb.set_trace() > 51 -> class RulerCanvas (Canvas): > 52 > 53 def __init__(self, master ): > 54 Canvas.__init__(self, master, width=200+RULER_LENGTH, height=100 ) > 55 self.draw_ruler() > 56 self.draw_well() > (Pdb) > > My problem is why the difference? and how to find out where that message comes from? > > PS. I also run this test under a Win8.1/64bit machine, get the same result. > PSS. I had scan files in the whole example directory and found there is > no any file using ttk module. Right. I got this with IDLE tests before using ttk. Good luck tracing this to its origin. -- Terry Jan Reedy From jfong at ms4.hinet.net Mon Jun 12 02:19:53 2017 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sun, 11 Jun 2017 23:19:53 -0700 (PDT) Subject: Strang thing in tkinter, and pdb too? In-Reply-To: References: <47eda4fc-02b8-4555-b8b2-b44ea8f4eebb@googlegroups.com> Message-ID: <65a981c7-cb75-4125-8795-38234548a770@googlegroups.com> Terry Reedy? 2017/06/12 UTC+8 12:04:18PM wrote: > Right. I got this with IDLE tests before using ttk. Good luck tracing > this to its origin. A little progress. If I remove temp.destroy() at line 34 then that message is gone. hmm...trying to find another way of doing it:-) --Jach Fong From auriocus at gmx.de Mon Jun 12 03:07:56 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Mon, 12 Jun 2017 09:07:56 +0200 Subject: Strang thing in tkinter, and pdb too? In-Reply-To: References: <47eda4fc-02b8-4555-b8b2-b44ea8f4eebb@googlegroups.com> Message-ID: Am 12.06.17 um 06:03 schrieb Terry Reedy: > On 6/11/2017 10:06 PM, jfong at ms4.hinet.net wrote: >> D:\Temp\widget-tour-py3>python canvasruler.py >> can't invoke "event" command: application has been destroyed >> while executing >> "event generate $w <>" >> (procedure "ttk::ThemeChanged" line 6) >> invoked from within >> "ttk::ThemeChanged" > > I have seen this too. It is not specific to any python/tkinter code, > but is specific to exiting with some things destroyed and something not. > This must be coming from tcl or tk, not python or tkinter. I can confirm that this is a Tk error message, with the following cause: In Tk, the root window with the name ".", which in Python comes to live with the Tk() function, determines the lifetime of a program. If you do: root=Tk() an empty window appears. If now the user closes this window with the "x" button on top, Tk stops - because usually this means that the program should stop - but Tcl does continue, in order to clean up stuff before leaving. If now further Tk commands are sent, they give the mentioned error: >>> import Tkinter >>> root=Tkinter.Tk() # here the window was closed using the "x" button >>> root # however the Tk() object still exists, # which is a Tcl interpreter with a dysfunct Tk >>> root.update() Traceback (most recent call last): File "", line 1, in File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1019, in update self.tk.call('update') _tkinter.TclError: can't invoke "update" command: application has been destroyed >>> This behaviour makes the root window special in the sense that the whole application stops, when it is closed, because closing this window exits the mainloop(). This makes sense for many applications, however sometimes it does not. If you wish to create many windows which pop up by themselves and the user can close them freely, then one should withdraw the main window: root=Tkinter.Tk() root.withdraw() now the main window disappears, the user can't incidentally close it. One can then create additional toplevel windows using t=Tkinter.Toplevel() and place anything there. These can be closed without destroying the application / exiting the main loop. Christian From ccubed2k4 at gmail.com Mon Jun 12 03:51:31 2017 From: ccubed2k4 at gmail.com (ccubed2k4 at gmail.com) Date: Mon, 12 Jun 2017 00:51:31 -0700 (PDT) Subject: Generate PDF with Tamil font problem In-Reply-To: References: Message-ID: <77a9f0b8-dc0a-4c8a-b5ad-9e82cd00a6a9@googlegroups.com> On Saturday, 5 March 2011 10:50:25 UTC+5:30, sath... at e-ndicus.com wrote: > Hi All, > > I am using python's reportlab to print some unicode Tamil characters > 'பே'. I added necessary unicode font to reportlab. But It > prints the output as 'ேப' (in reverse order). This issue > happens for multi-byte characters, whereas for character 'ப' is > printed as it is. > I am struggling to figure out the issue. Any help would see me on track. > > Thank you, > -- > SatheeshKumar. P > +91 99446 38595 > +91 87544 15303 > satheesh at e-ndicus.com > pskumar65 at gmail.com I am also facing the same problem with another similar Indian language- Malayalam. Has anybody encountered such issue and found any solution for this? From steve+python at pearwood.info Mon Jun 12 07:18:11 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 12 Jun 2017 21:18:11 +1000 Subject: Generate PDF with Tamil font problem References: <77a9f0b8-dc0a-4c8a-b5ad-9e82cd00a6a9@googlegroups.com> Message-ID: <593e7876$0$1619$c3e8da3$5496439d@news.astraweb.com> On Mon, 12 Jun 2017 05:51 pm, ccubed2k4 at gmail.com wrote: > On Saturday, 5 March 2011 10:50:25 UTC+5:30, sath... at e-ndicus.com wrote: >> Hi All, >> >> I am using python's reportlab to print some unicode Tamil characters >> 'பே'. I added necessary unicode font to reportlab. But It >> prints the output as 'ேப' (in reverse order). This issue >> happens for multi-byte characters, whereas for character 'ப' is >> printed as it is. >> I am struggling to figure out the issue. Any help would see me on track. >> >> Thank you, [...] > I am also facing the same problem with another similar Indian language- > Malayalam. Has anybody encountered such issue and found any solution for this? You are replying to a six year old message from 2011. I doubt anyone remembers the details of that discussion. Please show us a SIMPLE example of this issue. If I try this, I get something which appears to be okay: py> print('\u2986\u3015') ?? but they don't appear to be Tamil characters? py> for c in '\u2986\u3015': ... unicodedata.name(c) ... 'RIGHT WHITE PARENTHESIS' 'RIGHT TORTOISE SHELL BRACKET' Let me try this instead: py> for c in (chr(2986), chr(3015)): ... unicodedata.name(c) ... 'TAMIL LETTER PA' 'TAMIL VOWEL SIGN EE' That's better! py> print(''.join([chr(2986), chr(3015)])) ?? If you want help to diagnose and fix this, you will need to show us some simple code demonstrating the problem. Also tell us what output you expect, and what version of Python you are running. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From dbmarquand at gmail.com Mon Jun 12 10:47:31 2017 From: dbmarquand at gmail.com (David Marquand) Date: Mon, 12 Jun 2017 07:47:31 -0700 (PDT) Subject: Cannot get any Python commands to work Message-ID: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> I am trying to learn Django and cannot get easy_install to work. Python working on PyDev fine. PS C:\Windows\system32> python Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> easy_install --version Traceback (most recent call last): File "", line 1, in NameError: name 'easy_install' is not defined >>> python Traceback (most recent call last): File "", line 1, in NameError: name 'python' is not defined >>> From viktor.hagstrom at outlook.com Mon Jun 12 11:50:21 2017 From: viktor.hagstrom at outlook.com (=?utf-8?B?VmlrdG9yIEhhZ3N0csO2bQ==?=) Date: Mon, 12 Jun 2017 15:50:21 +0000 Subject: Cannot get any Python commands to work In-Reply-To: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> References: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> Message-ID: It seems like you are trying to run easy_install while running the Python interpreter. I am not familiar with easy_install, but my guess would be to run it outside the interpreter, in the command prompt. 2017-06-12 16:47 GMT+02:00 David Marquand >: I am trying to learn Django and cannot get easy_install to work. Python working on PyDev fine. PS C:\Windows\system32> python Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> easy_install --version Traceback (most recent call last): File "", line 1, in NameError: name 'easy_install' is not defined >>> python Traceback (most recent call last): File "", line 1, in NameError: name 'python' is not defined >>> -- https://mail.python.org/mailman/listinfo/python-list From viktor.hagstrom at outlook.com Mon Jun 12 11:50:25 2017 From: viktor.hagstrom at outlook.com (=?utf-8?B?VmlrdG9yIEhhZ3N0csO2bQ==?=) Date: Mon, 12 Jun 2017 15:50:25 +0000 Subject: Cannot get any Python commands to work In-Reply-To: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> References: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> Message-ID: It seems like you are trying to run easy_install while running the Python interpreter. I am not familiar with easy_install, but my guess would be to run it outside the interpreter, in the command prompt. 2017-06-12 16:47 GMT+02:00 David Marquand >: I am trying to learn Django and cannot get easy_install to work. Python working on PyDev fine. PS C:\Windows\system32> python Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> easy_install --version Traceback (most recent call last): File "", line 1, in NameError: name 'easy_install' is not defined >>> python Traceback (most recent call last): File "", line 1, in NameError: name 'python' is not defined >>> -- https://mail.python.org/mailman/listinfo/python-list From richardmoseley4 at gmail.com Mon Jun 12 12:34:27 2017 From: richardmoseley4 at gmail.com (Richard Moseley) Date: Mon, 12 Jun 2017 17:34:27 +0100 Subject: Cannot get any Python commands to work In-Reply-To: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> References: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> Message-ID: I believe that you need to be using this as a command rather a module from within python, in other words try executing easy_install from the command line, or try using pip which is now preferred method of installing new third-party packages. You may need to add '-3.6' or '-36' if you have multiple versions installed. On 12 June 2017 at 15:47, David Marquand wrote: > I am trying to learn Django and cannot get easy_install to work. Python > working on PyDev fine. > > > > > > > > > PS C:\Windows\system32> python > Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit > (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> easy_install --version > Traceback (most recent call last): > File "", line 1, in > NameError: name 'easy_install' is not defined > >>> python > Traceback (most recent call last): > File "", line 1, in > NameError: name 'python' is not defined > >>> > > -- > https://mail.python.org/mailman/listinfo/python-list > From uttamgoli1234 at gmail.com Mon Jun 12 13:39:07 2017 From: uttamgoli1234 at gmail.com (Forsaken uttax) Date: Mon, 12 Jun 2017 23:09:07 +0530 Subject: hello!! i was Installing Tensor flow with gpu support and something went wrong Message-ID: hello!! I was trying to install Tensorflow on my laptop with Gpu support so I Installed python the one in the screenshot below, and I put in pip Tensorflow Install command and it says syntax error so i try to update the pip that too doesn't work desperate for solution help me. Regards Teenage wonders From dbmarquand at gmail.com Mon Jun 12 14:06:04 2017 From: dbmarquand at gmail.com (David) Date: Mon, 12 Jun 2017 11:06:04 -0700 (PDT) Subject: Cannot get any Python commands to work In-Reply-To: References: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> Message-ID: <1b0d7372-3fe8-44b8-8b80-4b0ceb34208e@googlegroups.com> On Monday, June 12, 2017 at 11:34:41 AM UTC-5, Richard Moseley wrote: > I believe that you need to be using this as a command rather a module from > within python, in other words try executing easy_install from the command > line, or try using pip which is now preferred method of installing new > third-party packages. You may need to add '-3.6' or '-36' if you have > multiple versions installed. > > On 12 June 2017 at 15:47, David Marquand wrote: > > > I am trying to learn Django and cannot get easy_install to work. Python > > working on PyDev fine. > > > > > > > > > > > > > > > > > > PS C:\Windows\system32> python > > Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit > > (AMD64)] on win32 > > Type "help", "copyright", "credits" or "license" for more information. > > >>> easy_install --version > > Traceback (most recent call last): > > File "", line 1, in > > NameError: name 'easy_install' is not defined > > >>> python > > Traceback (most recent call last): > > File "", line 1, in > > NameError: name 'python' is not defined > > >>> > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > Along with some others, you are right and this worked. Thank you. David From jladasky at itu.edu Mon Jun 12 14:18:18 2017 From: jladasky at itu.edu (jladasky at itu.edu) Date: Mon, 12 Jun 2017 11:18:18 -0700 (PDT) Subject: hello!! i was Installing Tensor flow with gpu support and something went wrong In-Reply-To: References: Message-ID: <8db6807e-9098-43c9-88a5-a8700703d37b@googlegroups.com> On Monday, June 12, 2017 at 10:40:25 AM UTC-7, Forsaken uttax wrote: > hello!! > I was trying to install Tensorflow on my laptop with Gpu support > so I Installed python the one in the screenshot below, and I put in pip > Tensorflow Install command and it says syntax error so i try to update the > pip that too doesn't work desperate for solution help me. > > Regards > Teenage wonders You cannot attach a screen shot to the newsgroup. Building TensorFlow with GPU support is not simple. It involves far more than Python. I am investigating this issue myself. Check tensorflow.org for more information. Also, there is a TensorFlow newsgroup hosted on Google Groups. But before you take the trouble, are you sure that you need GPU support right away? You can explore neural nets on a CPU-only system, and in that configuration, TensorFlow installs easily. From josemsuarezsierra at gmail.com Mon Jun 12 14:32:51 2017 From: josemsuarezsierra at gmail.com (=?UTF-8?Q?Jos=C3=A9_Manuel_Su=C3=A1rez_Sierra?=) Date: Mon, 12 Jun 2017 11:32:51 -0700 (PDT) Subject: How to store some elements from a list into another Message-ID: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> Hello, I am stuck with a (perhaps) easy problem, I hope someone can help me: My problem is: I have a list of lists like this one: [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, 109, 110, 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, 136, 137, 138, 184, 185, 186, 187, 216, 217, 218, 219, 232, 233, 234, 235, 236, 237, 238, 267, 268, 269, 270, 272, 273, 274, 275], [2, 3, 4, 5, 9, 10, 11, 12, 21, 22, 23, 24, 29, 30, 31, 32, 56, 57, 58, 59, 65, 66, 67, 68, 74, 75, 76, 77, 78, 89, 90, 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, 128, 131, 132, 133, 134, 135]] And what I want is to store some of these datum into another list according to the next conditions: 1. I just want to store data if these values are consecutive (four in four), for instance, for first element I would like to store into the new list: [[[55,58],[83,86],....[n,n+3]]] and so on. I tried something like this: x=0 y=0 while list6[x][y] == list6[x][y+1]-1 and list6[x][y] == list6[x][y+1]-2 and list6[x][y] == list6[x][y+1]-3 or list6[x][0]: list7.append(list6[x][y]) list7.append(list6[x][y+3]) y = y+1 if (list6[x][y])%(list6[x][len(list6[x])-1]) == 0: x= x+1 if len(list6[x]) == x and len(list6[x][y]) == y: break It does not work I appreciate your help Thank you From python at mrabarnett.plus.com Mon Jun 12 14:54:16 2017 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 12 Jun 2017 19:54:16 +0100 Subject: How to store some elements from a list into another In-Reply-To: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> Message-ID: <943e6545-8533-e07c-87d6-329f74dbed8e@mrabarnett.plus.com> On 2017-06-12 19:32, Jos? Manuel Su?rez Sierra wrote: > Hello, > I am stuck with a (perhaps) easy problem, I hope someone can help me: > > My problem is: > I have a list of lists like this one: > [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, 109, 110, 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, 136, 137, 138, 184, 185, 186, 187, 216, 217, 218, 219, 232, 233, 234, 235, 236, 237, 238, 267, 268, 269, 270, 272, 273, 274, 275], [2, 3, 4, 5, 9, 10, 11, 12, 21, 22, 23, 24, 29, 30, 31, 32, 56, 57, 58, 59, 65, 66, 67, 68, 74, 75, 76, 77, 78, 89, 90, 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, 128, 131, 132, 133, 134, 135]] > > And what I want is to store some of these datum into another list according to the next conditions: > 1. I just want to store data if these values are consecutive (four in four), for instance, for first element I would like to store into the new list: [[[55,58],[83,86],....[n,n+3]]] and so on. > I tried something like this: > > x=0 > y=0 Here you're checking whether list6[x][y+1] is list6[x][y]+1 _and_ list6[x][y]+2 _and_ list6[x][y]+3, all at the same time. That's not going to happen! > while list6[x][y] == list6[x][y+1]-1 and list6[x][y] == list6[x][y+1]-2 and list6[x][y] == list6[x][y+1]-3 or list6[x][0]: > > list7.append(list6[x][y]) > list7.append(list6[x][y+3]) > y = y+1 > if (list6[x][y])%(list6[x][len(list6[x])-1]) == 0: > x= x+1 > > if len(list6[x]) == x and len(list6[x][y]) == y: > break > > > It does not work > > I appreciate your help > Thank you > From tjol at tjol.eu Mon Jun 12 15:09:31 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 12 Jun 2017 21:09:31 +0200 Subject: How to store some elements from a list into another In-Reply-To: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> Message-ID: <593ee705$0$1815$e4fe514c@news.kpn.nl> On 12/06/17 20:32, Jos? Manuel Su?rez Sierra wrote: > Hello, > I am stuck with a (perhaps) easy problem, I hope someone can help me: > > My problem is: > I have a list of lists like this one: > [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, 109, 110, 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, 136, 137, 138, 184, 185, 186, 187, 216, 217, 218, 219, 232, 233, 234, 235, 236, 237, 238, 267, 268, 269, 270, 272, 273, 274, 275], [2, 3, 4, 5, 9, 10, 11, 12, 21, 22, 23, 24, 29, 30, 31, 32, 56, 57, 58, 59, 65, 66, 67, 68, 74, 75, 76, 77, 78, 89, 90, 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, 128, 131, 132, 133, 134, 135]] > > And what I want is to store some of these datum into another list according to the next conditions: > 1. I just want to store data if these values are consecutive (four in four), for instance, for first element I would like to store into the new list: [[[55,58],[83,86],....[n,n+3]]] and so on. > I tried something like this: Something similar to this should work with a few fixes (though it won't be very "pythonic"). Let's look at what this code is doing! > > x=0 > y=0 > while list6[x][y] == list6[x][y+1]-1 and list6[x][y] == list6[x][y+1]-2 and list6[x][y] == list6[x][y+1]-3 or list6[x][0]: Right now, x=0, y=0, and I assume list6[0] is [55, 56, 57, 58, 83, ...] so: - list6[x][y] == list6[x][y+1]-1 is True: 55 == 56-1 - list6[x][y] == list6[x][y+1]-2 is False: 55 != 56-2 - list6[x][y] == list6[x][y+1]-3 is False: 55 != 56-3 - list6[x][0] is True: 55 != 0. So the entire expression is equivalent to (at this moment: while True and False and False or True: which is equivalent to while True. (Do you see the problem with your while statement?) > > list7.append(list6[x][y]) > list7.append(list6[x][y+3]) We enter the look and things are added to your list. Anyway, I hope you see the mistake you made in your while condition. > y = y+1 Are you sure you want to increase y by one only? > if (list6[x][y])%(list6[x][len(list6[x])-1]) == 0: > x= x+1 > > if len(list6[x]) == x and len(list6[x][y]) == y: > break > > > It does not work You should be able to figure out why it doesn't work (and fix it!) by carefully checking what each step does. It's often helpful to add print() calls at strategic places in your code to check what exactly it's doing, and track what's going on. For example: py> list6 = [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, 109, 110, 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, 136, 137, 138, 184, 185, 186, 187, 216, 217, 218, 219, 232, 233, 234, 235, 236, 237, 238, 267, 268, 269, 270, 272, 273, 274, 275], [2, 3, 4, 5, 9, 10, 11, 12, 21, 22, 23, 24, 29, 30, 31, 32, 56, 57, 58, 59, 65, 66, 67, 68, 74, 75, 76, 77, 78, 89, 90, 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, 128, 131, 132, 133, 134, 135]] py> list7 = [] py> x=0 py> y=0 py> while list6[x][y] == list6[x][y+1]-1 and list6[x][y] == list6[x][y+1]-2 and list6[x][y] == list6[x][y+1]-3 or list6[x][0]: ... list7.append(list6[x][y]) ... list7.append(list6[x][y+3]) ... print('appended', list6[x][y], list6[x][y+3]) ... y = y+1 ... if (list6[x][y])%(list6[x][len(list6[x])-1]) == 0: ... x = x+1 ... if len(list6[x]) == x and len(list6[x][y]) == y: ... break ... appended 55 58 appended 56 83 appended 57 84 [...] Traceback (most recent call last): File "", line 3, in IndexError: list index out of range py> list7 [55, 58, 56, 83, 57, 84, 58, 85, 83, 86, 84, 89, 85, 90, 86, 91, 89, 92, 90, 107, 91, 108, 92, 109, 107, 110, 108, 111, 109, 117, 110, 118, 111, 119, 117, 120, 118, 128, 119, 129, 120, 130, 128, 131, 129, 135, 130, 136, 131, 137, 135, 138, 136, 184, 137, 185, 138, 186, 184, 187, 185, 216, 186, 217, 187, 218, 216, 219, 217, 232, 218, 233, 219, 234, 232, 235, 233, 236, 234, 237, 235, 238, 236, 267, 237, 268, 238, 269, 267, 270, 268, 272, 269, 273, 270, 274, 272, 275, 273] py> I hope these hints were of some help. I'll leave the next steps as an exercise for you (learning to debug is important, and hard!). If you come back when you've made some more progress, (or got it working) I'll show you some more elegant solutions (unless someone else beats me to it, of course). Good luck Thomas > > I appreciate your help > Thank you > From tjreedy at udel.edu Mon Jun 12 16:15:20 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 Jun 2017 16:15:20 -0400 Subject: hello!! i was Installing Tensor flow with gpu support and something went wrong In-Reply-To: <8db6807e-9098-43c9-88a5-a8700703d37b@googlegroups.com> References: <8db6807e-9098-43c9-88a5-a8700703d37b@googlegroups.com> Message-ID: On 6/12/2017 2:18 PM, jladasky at itu.edu wrote: > On Monday, June 12, 2017 at 10:40:25 AM UTC-7, Forsaken uttax wrote: >> hello!! >> I was trying to install Tensorflow on my laptop with Gpu support >> so I Installed python the one in the screenshot below, and I put in pip >> Tensorflow Install command and it says syntax error so i try to update the >> pip that too doesn't work desperate for solution help me. >> >> Regards >> Teenage wonders > > You cannot attach a screen shot to the newsgroup. > > Building TensorFlow with GPU support is not simple. It involves far more than Python. I am investigating this issue myself. Check tensorflow.org for more information. Also, there is a TensorFlow newsgroup hosted on Google Groups. There is also a tensorflow tag on Stackoverflow with lots of questions and, I hope, helpful answers. > But before you take the trouble, are you sure that you need GPU support right away? You can explore neural nets on a CPU-only system, and in that configuration, TensorFlow installs easily. > -- Terry Jan Reedy From ian.g.kelly at gmail.com Mon Jun 12 18:28:39 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 12 Jun 2017 16:28:39 -0600 Subject: Cannot get any Python commands to work In-Reply-To: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> References: <6cb3d645-7f02-42b8-be01-adf4ed6697c3@googlegroups.com> Message-ID: On Mon, Jun 12, 2017 at 8:47 AM, David Marquand wrote: > I am trying to learn Django and cannot get easy_install to work. Python working on PyDev fine. > > > PS C:\Windows\system32> python > Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> easy_install --version > Traceback (most recent call last): > File "", line 1, in > NameError: name 'easy_install' is not defined >>>> python > Traceback (most recent call last): > File "", line 1, in > NameError: name 'python' is not defined You need to run these from your regular OS shell, not from the Python interpeter. From ben+python at benfinney.id.au Mon Jun 12 21:57:11 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 13 Jun 2017 11:57:11 +1000 Subject: Customise the virtualenv `activate` script Message-ID: <85mv9cwsy0.fsf@benfinney.id.au> Howdy all, What is the conventional way to customise the changes made by a Python virtualenv bin/activate script? The Python virtualenv is activated by a ?$VENV/bin/activate script? [0]. This script works primarily by setting environment variables specific to the virtualenv. (This is different from the ?bootstrap script? discussed in the Virtualenv documentation. Those are used once, when creating the virtualenv; they are not used when activating the virtualenv. This question is about the activate script.) Many of the code bases for which I use a Python virtualenv, need additional (custom) environment variables set, each time the virtualenv is activated. How can I make this easy for developers who already know how to activate a virtualenv? * Edit the ?$VENV/bin/activate? script directly, to add statements that set more environment variables? * Write a custom wrapper script, that incidentally calls ?$VENV/bin/activate?? * Write a separate script with a specific name, that will be automatically called by ?$VENV/bin/activate?? Does such a thing exist? Of course I could write a script with a different name, and instruct developers to run that instead. Instead, this question is asking how to hook into the existing convention, of activating the virtualenv with a known name ?$VENV/bin/activate?. [0] https://virtualenv.pypa.io/en/stable/userguide/#activate -- \ ?It is clear that thought is not free if the profession of | `\ certain opinions makes it impossible to earn a living.? | _o__) ?Bertrand Russell, _Free Thought and Official Propaganda_, 1928 | Ben Finney From breamoreboy at gmail.com Mon Jun 12 22:37:00 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 12 Jun 2017 19:37:00 -0700 (PDT) Subject: How to store some elements from a list into another In-Reply-To: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> Message-ID: <34a1e967-cb36-424c-9b3e-2aa44e36212c@googlegroups.com> On Monday, June 12, 2017 at 7:33:03 PM UTC+1, Jos? Manuel Su?rez Sierra wrote: > Hello, > I am stuck with a (perhaps) easy problem, I hope someone can help me: > > My problem is: > I have a list of lists like this one: > [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, 109, 110, 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, 136, 137, 138, 184, 185, 186, 187, 216, 217, 218, 219, 232, 233, 234, 235, 236, 237, 238, 267, 268, 269, 270, 272, 273, 274, 275], [2, 3, 4, 5, 9, 10, 11, 12, 21, 22, 23, 24, 29, 30, 31, 32, 56, 57, 58, 59, 65, 66, 67, 68, 74, 75, 76, 77, 78, 89, 90, 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, 128, 131, 132, 133, 134, 135]] > > And what I want is to store some of these datum into another list according to the next conditions: > 1. I just want to store data if these values are consecutive (four in four), for instance, for first element I would like to store into the new list: [[[55,58],[83,86],....[n,n+3]]] and so on. > I tried something like this: > > x=0 > y=0 > while list6[x][y] == list6[x][y+1]-1 and list6[x][y] == list6[x][y+1]-2 and list6[x][y] == list6[x][y+1]-3 or list6[x][0]: > > list7.append(list6[x][y]) > list7.append(list6[x][y+3]) > y = y+1 > if (list6[x][y])%(list6[x][len(list6[x])-1]) == 0: > x= x+1 > > if len(list6[x]) == x and len(list6[x][y]) == y: > break > > > It does not work > > I appreciate your help > Thank you Perhaps use the recipe for consecutive numbers from here https://docs.python.org/2.6/library/itertools.html#examples It will have to be modified for Python 3, I'll leave that as an exercise. Kindest regards. Mark Lawrence. From mrbm74 at gmail.com Tue Jun 13 01:02:16 2017 From: mrbm74 at gmail.com (Martin Bammer) Date: Mon, 12 Jun 2017 22:02:16 -0700 (PDT) Subject: Why is it faster to first pickle data before sending a dict than sending a dict directly in Python 3? Message-ID: Following test program: import time import marshal try: import cPickle as pickle str_ = unicode except: import pickle str_ = str def TestPrc(rd): d = rd.recv() print("OK") def TestPrc2(rd): d = pickle.loads(rd.recv()) print("OK") if __name__ == "__main__": from multiprocessing import freeze_support, Process, Pipe freeze_support() d = { str_(x) : u"Hello World" + str_(x + 1) for x in range(1000000) } wr, rd = Pipe() p = Process(target = TestPrc, args = ( rd, )) p.start() t1 = time.time() wr.send(d) print("#1", time.time() - t1) p.join() print("#2", time.time() - t1) p = Process(target = TestPrc2, args = ( rd, )) p.start() t1 = time.time() wr.send(pickle.dumps(d, pickle.HIGHEST_PROTOCOL)) print("#3", time.time() - t1) p.join() print("#4", time.time() - t1) I get the following results: Python 2.7: ('#1', 0.33500003814697266) OK ('#2', 0.7890000343322754) ('#3', 0.36300015449523926) OK ('#4', 0.8059999942779541) Python 3.4: #1 0.7770781517028809 OK #2 1.4451451301574707 #3 0.7410738468170166 OK #4 1.3691368103027344 Python 3.6: #1 0.681999921798706 OK #2 1.1500000953674316 #3 0.6549999713897705 OK #4 1.1089999675750732 The results show that in Python 3 it is faster to do the (un-)pickling in the Python code. I would expect to have no real performance difference, or at least the more straight forward way to send Python objects directly to be a bit faster, as it is in Python 2. Some other interesting results from this example: - Python 2 is much faster - At least Python 3.6 is much faster than Python 3.4 Regards, Martin From jussi.piitulainen at helsinki.fi Tue Jun 13 03:48:53 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Tue, 13 Jun 2017 10:48:53 +0300 Subject: How to store some elements from a list into another References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> <34a1e967-cb36-424c-9b3e-2aa44e36212c@googlegroups.com> Message-ID: breamoreboy at gmail.com writes: > On Monday, June 12, 2017 at 7:33:03 PM UTC+1, Jos? Manuel Su?rez Sierra wrote: >> Hello, >> I am stuck with a (perhaps) easy problem, I hope someone can help me: >> >> My problem is: >> I have a list of lists like this one: >> [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, 109, 110, >> 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, 136, 137, 138, 184, >> 185, 186, 187, 216, 217, 218, 219, 232, 233, 234, 235, 236, 237, 238, >> 267, 268, 269, 270, 272, 273, 274, 275], [2, 3, 4, 5, 9, 10, 11, 12, >> 21, 22, 23, 24, 29, 30, 31, 32, 56, 57, 58, 59, 65, 66, 67, 68, 74, >> 75, 76, 77, 78, 89, 90, 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, >> 128, 131, 132, 133, 134, 135]] >> >> And what I want is to store some of these datum into another list >> according to the next conditions: >> >> 1. I just want to store data if these values are consecutive (four in >> four), for instance, for first element I would like to store into the >> new list: [[[55,58],[83,86],....[n,n+3]]] and so on. >> >> I tried something like this: >> >> x=0 >> y=0 >> while list6[x][y] == list6[x][y+1]-1 and list6[x][y] == list6[x][y+1]-2 and list6[x][y] == list6[x][y+1]-3 or list6[x][0]: >> >> list7.append(list6[x][y]) >> list7.append(list6[x][y+3]) >> y = y+1 >> if (list6[x][y])%(list6[x][len(list6[x])-1]) == 0: >> x= x+1 >> >> if len(list6[x]) == x and len(list6[x][y]) == y: >> break >> >> >> It does not work >> >> I appreciate your help >> Thank you > > Perhaps use the recipe for consecutive numbers from here > https://docs.python.org/2.6/library/itertools.html#examples It will > have to be modified for Python 3, I'll leave that as an exercise. What a clever idea. Pity it's gone in newer documentation. (By the "it" in the previous sentence I refer only to the idea of grouping by the difference to the index in the original sequence, and by "gone" only to the fact that I didn't see this example at the corresponding location for Python 3.6, which I found by replacing the 2 in the URL with 3. Perhaps the idea is preserved somewhere else?) Anyway, I've adapted it to Python 3, and to an analysis of the problem at hand - mainly the problem that the OP finds themselves _stuck_ with their spec and their code, as quoted above. Hope it helps. What follows, follows. # The big idea is to define (auxiliary) functions. It's not an # advanced idea. It's the most basic of all ideas. The experience of # being stuck comes from trying to see the whole problem at once. # Ok, familiary with standard ways of viewing things helps. But that # is just the flip side of breaking problems into manageable parts: # past experience suggests that some kinds of parts are more useful, # more composable into a solution, so in standard libraries. # One subproblem is to group just one list of numbers, then it is easy # to group every list in a list of such lists. But another subproblem # is to deal with one group of numbers. There seem to be two concerns: # a group should consist of consecutive numbers, and a group should # consist of four numbers - the latter at least is easy enough if the # group is stored as a list, but what should be done if there are five # or seven numbers? No idea, but that can be clarified later once the # components of a solution are untangled into their own functions. # The battle cry is: Use def! import itertools as it import operator as op def applied(f): '''Reification of that asterisk - like a really advanced computer-sciency kind of thing. But see no lambda!''' def F(args): return f(*args) return F def consequences(data): '''Lists of consecutive datami, clever idea adapted from https://docs.python.org/2.6/library/itertools.html#examples''' for k, group in it.groupby(enumerate(data), applied(op.sub)): yield [datum for index, datum in group] def isfourlong(sequence): '''True if sequence is of length 4.''' return len(sequence) == 4 def ends(sequences): '''Collect the endpoints of sequences in a list of 2-lists.''' return [[sequence[0], sequence[-1]] for sequence in sequences] data = [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, 109, 110, 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, 136, 137, 138, 184, 185, 186, 187, 216, 217, 218, 219, 232, 233, 234, 235, 236, 237, 238, 267, 268, 269, 270, 272, 273, 274, 275], [2, 3, 4, 5, 9, 10, 11, 12, 21, 22, 23, 24, 29, 30, 31, 32, 56, 57, 58, 59, 65, 66, 67, 68, 74, 75, 76, 77, 78, 89, 90, 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, 128, 131, 132, 133, 134, 135]] def testrun(): '''See how many variations can be composed out of the few auxiliary functions - the problem becomes tame, or at least a bit tamer. This kind of ad-hoc test suite is very useful, during development, when at all in doubt.''' print('groups in both lists:') print(list(consequences(data[0]))) print(list(consequences(data[1]))) # note the calls to list() - consequences() returns an opaque # generator object that can be drained into a list when needed; # filter objects below are similarly opaque and drainable - define # an auxiliary that returns a list instead if that feels scary - # Don't Panic! print() print('groups of four in both lists:') print(list(filter(isfourlong, consequences(data[0])))) print(list(filter(isfourlong, consequences(data[1])))) print() print('ends of all groups in both lists:') print(ends(consequences(data[0]))) print(ends(consequences(data[1]))) print() print('ends of groups of four in both lists:') print(ends(filter(isfourlong, consequences(data[0])))) print(ends(filter(isfourlong, consequences(data[1])))) print() print('lists of ends of all groups:') print([ends(consequences(datami)) for datami in data]) print() print('lists of ends of groups of four:') print([ends(filter(isfourlong, consequences(datami))) for datami in data]) if __name__ == '__main__': # inside the above condition, these commands are executed when the # script is executed but not when it is imported as a library, so # it is easy to run the test from the shell and easy to import the # definitions into an interactive session testrun() From cs at zip.com.au Tue Jun 13 03:50:57 2017 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 13 Jun 2017 17:50:57 +1000 Subject: Customise the virtualenv `activate` script In-Reply-To: <85mv9cwsy0.fsf@benfinney.id.au> References: <85mv9cwsy0.fsf@benfinney.id.au> Message-ID: <20170613075057.GA51505@cskk.homeip.net> On 13Jun2017 11:57, Ben Finney wrote: >Many of the code bases for which I use a Python virtualenv, need >additional (custom) environment variables set, each time the virtualenv >is activated. > >How can I make this easy for developers who already know how to activate >a virtualenv? > >* Edit the ?$VENV/bin/activate? script directly, to add statements that > set more environment variables? > >* Write a custom wrapper script, that incidentally calls > ?$VENV/bin/activate?? > >* Write a separate script with a specific name, that will be > automatically called by ?$VENV/bin/activate?? Does such a thing exist? > >Of course I could write a script with a different name, and instruct >developers to run that instead. Instead, this question is asking how to >hook into the existing convention, of activating the virtualenv with a >known name ?$VENV/bin/activate?. I cannot speak for the conventions. I must admit my initial preference would be the differently named wrapper. Surely users of the codebase will be invoking stuff via something opaque which sources the requisite things? Actually, on trying to write something simple and flexible, since once made the venv is basicly state WRT the activate script, I'm leaning towards hacking the activate script, probably by keeping a distinct file off the the side and modifying activate to source it. Cheers, Cameron Simpson From __peter__ at web.de Tue Jun 13 06:00:30 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 13 Jun 2017 12:00:30 +0200 Subject: How to store some elements from a list into another References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> <34a1e967-cb36-424c-9b3e-2aa44e36212c@googlegroups.com> Message-ID: Jussi Piitulainen wrote: > breamoreboy at gmail.com writes: > >> On Monday, June 12, 2017 at 7:33:03 PM UTC+1, Jos? Manuel Su?rez Sierra >> wrote: >>> Hello, >>> I am stuck with a (perhaps) easy problem, I hope someone can help me: >>> >>> My problem is: >>> I have a list of lists like this one: >>> [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, 109, 110, >>> 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, 136, 137, 138, 184, >>> 185, 186, 187, 216, 217, 218, 219, 232, 233, 234, 235, 236, 237, 238, >>> 267, 268, 269, 270, 272, 273, 274, 275], [2, 3, 4, 5, 9, 10, 11, 12, >>> 21, 22, 23, 24, 29, 30, 31, 32, 56, 57, 58, 59, 65, 66, 67, 68, 74, >>> 75, 76, 77, 78, 89, 90, 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, >>> 128, 131, 132, 133, 134, 135]] >>> >>> And what I want is to store some of these datum into another list >>> according to the next conditions: >>> >>> 1. I just want to store data if these values are consecutive (four in >>> four), for instance, for first element I would like to store into the >>> new list: [[[55,58],[83,86],....[n,n+3]]] and so on. >>> >>> I tried something like this: >>> >>> x=0 >>> y=0 >>> while list6[x][y] == list6[x][y+1]-1 and list6[x][y] == >>> list6[x][y+1]-2 and list6[x][y] == list6[x][y+1]-3 or >>> list6[x][0]: >>> >>> list7.append(list6[x][y]) >>> list7.append(list6[x][y+3]) >>> y = y+1 >>> if (list6[x][y])%(list6[x][len(list6[x])-1]) == 0: >>> x= x+1 >>> >>> if len(list6[x]) == x and len(list6[x][y]) == y: >>> break >>> >>> >>> It does not work >>> >>> I appreciate your help >>> Thank you >> >> Perhaps use the recipe for consecutive numbers from here >> https://docs.python.org/2.6/library/itertools.html#examples It will >> have to be modified for Python 3, I'll leave that as an exercise. > > What a clever idea. Pity it's gone in newer documentation. (By the "it" > in the previous sentence I refer only to the idea of grouping by the > difference to the index in the original sequence, and by "gone" only to > the fact that I didn't see this example at the corresponding location > for Python 3.6, which I found by replacing the 2 in the URL with 3. > Perhaps the idea is preserved somewhere else?) > > Anyway, I've adapted it to Python 3, and to an analysis of the problem > at hand - mainly the problem that the OP finds themselves _stuck_ with > their spec and their code, as quoted above. Hope it helps. > > What follows, follows. > > # The big idea is to define (auxiliary) functions. It's not an > # advanced idea. It's the most basic of all ideas. The experience of > # being stuck comes from trying to see the whole problem at once. > > # Ok, familiary with standard ways of viewing things helps. But that > # is just the flip side of breaking problems into manageable parts: > # past experience suggests that some kinds of parts are more useful, > # more composable into a solution, so in standard libraries. > > # One subproblem is to group just one list of numbers, then it is easy > # to group every list in a list of such lists. But another subproblem > # is to deal with one group of numbers. There seem to be two concerns: > # a group should consist of consecutive numbers, and a group should > # consist of four numbers - the latter at least is easy enough if the > # group is stored as a list, but what should be done if there are five > # or seven numbers? No idea, but that can be clarified later once the > # components of a solution are untangled into their own functions. > > # The battle cry is: Use def! > > import itertools as it > import operator as op > > def applied(f): > '''Reification of that asterisk - like a really advanced > computer-sciency kind of thing. But see no lambda!''' > def F(args): return f(*args) > return F > > def consequences(data): > '''Lists of consecutive datami, clever idea adapted from > https://docs.python.org/2.6/library/itertools.html#examples''' > for k, group in it.groupby(enumerate(data), applied(op.sub)): > yield [datum for index, datum in group] Hm, the itertools users' code of honour requires that no intermediate sequences be materialised ;) So: second = op.itemgetter(1) def consequences(data): for k, group in it.groupby(enumerate(data), applied(op.sub)): yield edges(map(second, group)) def isfourlong(pair): min, max = pair return max - min == 3 def edges(items): first = last = next(items) for last in items: pass return [first, last] def ends(sequences): return list(sequences) However, this is infested with for loops. Therefore def consequences(data): groups = map(second, it.groupby(enumerate(data), applied(op.sub))) sans_index = map(functools.partial(map, second), groups) return map(edges, sans_index) and because we want to hide traces that this was written by mere mortals def consequences(data): return map( edges, map( functools.partial(map, second), map(second, it.groupby(enumerate(data), applied(op.sub))) ) ) I don't immediately see what to do about the for loop in edges(), so I'll use the traditional cop-out: Removing the last loop is left as an exercise... > def isfourlong(sequence): > '''True if sequence is of length 4.''' > return len(sequence) == 4 > > def ends(sequences): > '''Collect the endpoints of sequences in a list of 2-lists.''' > return [[sequence[0], sequence[-1]] for sequence in sequences] > > data = [[55, 56, 57, 58, 83, 84, 85, 86, 89, 90, 91, 92, 107, 108, > 109, 110, 111, 117, 118, 119, 120, 128, 129, 130, 131, 135, > 136, 137, 138, 184, 185, 186, 187, 216, 217, 218, 219, 232, > 233, 234, 235, 236, 237, 238, 267, 268, 269, 270, 272, 273, > 274, 275], > > [2, 3, 4, 5, 9, 10, 11, 12, 21, 22, 23, 24, 29, 30, 31, 32, > 56, 57, 58, 59, 65, 66, 67, 68, 74, 75, 76, 77, 78, 89, 90, > 91, 92, 98, 99, 100, 101, 102, 125, 126, 127, 128, 131, 132, > 133, 134, 135]] > > def testrun(): > '''See how many variations can be composed out of the few auxiliary > functions - the problem becomes tame, or at least a bit tamer. > This kind of ad-hoc test suite is very useful, during development, > when at all in doubt.''' > print('groups in both lists:') > print(list(consequences(data[0]))) > print(list(consequences(data[1]))) > # note the calls to list() - consequences() returns an opaque > # generator object that can be drained into a list when needed; > # filter objects below are similarly opaque and drainable - define > # an auxiliary that returns a list instead if that feels scary - > # Don't Panic! > print() > print('groups of four in both lists:') > print(list(filter(isfourlong, consequences(data[0])))) > print(list(filter(isfourlong, consequences(data[1])))) > print() > print('ends of all groups in both lists:') > print(ends(consequences(data[0]))) > print(ends(consequences(data[1]))) > print() > print('ends of groups of four in both lists:') > print(ends(filter(isfourlong, consequences(data[0])))) > print(ends(filter(isfourlong, consequences(data[1])))) > print() > print('lists of ends of all groups:') > print([ends(consequences(datami)) for datami in data]) > print() > print('lists of ends of groups of four:') > print([ends(filter(isfourlong, consequences(datami))) > for datami in data]) > > if __name__ == '__main__': > # inside the above condition, these commands are executed when the > # script is executed but not when it is imported as a library, so > # it is easy to run the test from the shell and easy to import the > # definitions into an interactive session > testrun() PS: Users who did not like my suggestions above may like the alternative shown below: def compress(values): """ >>> list(compress([])) [] >>> list(compress([1])) [(1, 1)] >>> list(compress([1, 2])) [(1, 2)] >>> list(compress([1, 2, 3])) [(1, 3)] >>> list(compress([1, 2, 3, 7, 8])) [(1, 3), (7, 8)] >>> list(compress([1, 2, 3, 7, 9, 10])) [(1, 3), (7, 7), (9, 10)] >>> list(compress([3, 4, 5, 1, 2, 3, 10, 11])) [(3, 5), (1, 3), (10, 11)] >>> list(compress([1, 1])) [(1, 1), (1, 1)] >>> list(compress([1,1,2,2,3,4,7,7])) [(1, 1), (1, 2), (2, 4), (7, 7), (7, 7)] """ values = iter(values) try: first = prev = value = next(values) except StopIteration: return for value in values: if value - prev != 1: yield first, prev first = value prev = value yield first, value From jussi.piitulainen at helsinki.fi Tue Jun 13 09:24:38 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Tue, 13 Jun 2017 16:24:38 +0300 Subject: How to store some elements from a list into another References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> <34a1e967-cb36-424c-9b3e-2aa44e36212c@googlegroups.com> Message-ID: Peter Otten writes: ... > def edges(items): > first = last = next(items) > for last in items: > pass > return [first, last] ... > However, this is infested with for loops. Therefore ... > I don't immediately see what to do about the for loop in edges(), so > I'll use the traditional cop-out: Removing the last loop is left as an > exercise... In the spirit of the exercise: def sekond(x, y): return y def edges(items): # where items is a non-empty iterator first = next(items) last = functools.reduce(sekond, items, first) return [first, last] Of course, right? From saxri89 at gmail.com Tue Jun 13 09:49:16 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Tue, 13 Jun 2017 06:49:16 -0700 (PDT) Subject: How to decompile an exe file compiled by py2exe? Message-ID: hello How to decompile an exe file compiled by py2exe? in my file i have a library.zip file i dont if that help this work. i need some easy because i am very new i try some programs but without results. From irving.duran at gmail.com Tue Jun 13 11:45:19 2017 From: irving.duran at gmail.com (Irving Duran) Date: Tue, 13 Jun 2017 15:45:19 +0000 Subject: How to decompile an exe file compiled by py2exe? In-Reply-To: References: Message-ID: This might be what you are looking for -> https://stackoverflow.com/questions/6287918/how-to-decompile-an-exe-file-compiled-by-py2exe On Tue, Jun 13, 2017 at 8:52 AM Xristos Xristoou wrote: > hello > > How to decompile an exe file compiled by py2exe? > in my file i have a library.zip file i dont if that help this work. > > i need some easy because i am very new i try some programs but without > results. > -- > https://mail.python.org/mailman/listinfo/python-list > -- Thank You, Irving Duran From robin at reportlab.com Tue Jun 13 12:09:52 2017 From: robin at reportlab.com (Robin Becker) Date: Tue, 13 Jun 2017 17:09:52 +0100 Subject: Progress on the Gilectomy In-Reply-To: <593ce2ee$0$1617$c3e8da3$5496439d@news.astraweb.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <593ce2ee$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 11/06/2017 07:27, Steve D'Aprano wrote: ........ > > I'm tired of people complaining about the GIL as a "mistake" without > acknowledging that it exists for a reason. > > > I thought we were also consenting adults about problems arising from bad extensions. The GIL is a blocker for cpython's ability to use multi-core cpus. I looked at Larry's talk with interest. The GIL is not a requirement as he pointed out at the end, both IronPython and Jython don't need it. That said I think the approach he outlined is probably wrong unless we attach a very high weight to preserving the current extension interface. C extensions are a real nuisance. The contention issues all arise from reference counting. Newer languages like go seem to prefer the garbage collection approach. Perhaps someone should try a reference-countectomy, but then they already have with other python implementations. -- Robin Becker From robin at reportlab.com Tue Jun 13 12:09:52 2017 From: robin at reportlab.com (Robin Becker) Date: Tue, 13 Jun 2017 17:09:52 +0100 Subject: Progress on the Gilectomy In-Reply-To: <593ce2ee$0$1617$c3e8da3$5496439d@news.astraweb.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <593ce2ee$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 11/06/2017 07:27, Steve D'Aprano wrote: ........ > > I'm tired of people complaining about the GIL as a "mistake" without > acknowledging that it exists for a reason. > > > I thought we were also consenting adults about problems arising from bad extensions. The GIL is a blocker for cpython's ability to use multi-core cpus. I looked at Larry's talk with interest. The GIL is not a requirement as he pointed out at the end, both IronPython and Jython don't need it. That said I think the approach he outlined is probably wrong unless we attach a very high weight to preserving the current extension interface. C extensions are a real nuisance. The contention issues all arise from reference counting. Newer languages like go seem to prefer the garbage collection approach. Perhaps someone should try a reference-countectomy, but then they already have with other python implementations. -- Robin Becker From skip.montanaro at gmail.com Tue Jun 13 13:26:03 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 13 Jun 2017 12:26:03 -0500 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <593ce2ee$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Jun 13, 2017 at 11:09 AM, Robin Becker wrote: > I looked at Larry's talk with interest. The GIL is not a requirement as he > pointed out at the end, both IronPython and Jython don't need it. But they don't support CPython's extension module API either, I don't think. (I imagine that might have been the point of your reference.) Skip From tjreedy at udel.edu Tue Jun 13 14:53:02 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 13 Jun 2017 14:53:02 -0400 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <593ce2ee$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 6/13/2017 12:09 PM, Robin Becker wrote: > On 11/06/2017 07:27, Steve D'Aprano wrote: > ........ >> >> I'm tired of people complaining about the GIL as a "mistake" without >> acknowledging that it exists for a reason. >> > I thought we were also consenting adults about problems arising from bad > extensions. The GIL is a blocker for cpython's ability to use multi-core > cpus. When using threads, not when using multiple processes. > The contention issues all arise from reference counting. Newer > languages like go seem to prefer the garbage collection approach. > Perhaps someone should try a reference-countectomy, This was tried at least once, perhaps 15 years ago. -- Terry Jan Reedy From skip.montanaro at gmail.com Tue Jun 13 15:09:05 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 13 Jun 2017 14:09:05 -0500 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <593ce2ee$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Jun 13, 2017 at 1:53 PM, Terry Reedy wrote: > This was tried at least once, perhaps 15 years ago. Yes, I believe Greg Smith (?) implemented a proof-of-concept in about the Python 1.4 timeframe. The observation at the time was that it slowed down single-threaded programs too much to be accepted as it existed then. That remains the primary bugaboo as I understand it. It seems Larry has pushed the envelope a fair bit farther, but there are still problems. I don't know if the Gilectomy code changes are too great to live along the mainline branches, but I wonder if having a bleeding-edge-gilectomy branch in Git (maintained alongside the regular stuff, but not formally released) would a) help it stay in sync better with CPython b) expose the changes to more people, especially extension module authors Combined, the two might make it so the GIL-free branch isn't always playing catchup (because of 'a') and more extension modules get tweaked to work properly in a GIL-free world (because of 'b'). I imagine Larry Hastings has given the idea some consideration. Skip From __peter__ at web.de Tue Jun 13 17:19:44 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 13 Jun 2017 23:19:44 +0200 Subject: How to store some elements from a list into another References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> <34a1e967-cb36-424c-9b3e-2aa44e36212c@googlegroups.com> Message-ID: Jussi Piitulainen wrote: > Peter Otten writes: > > ... > >> def edges(items): >> first = last = next(items) >> for last in items: >> pass >> return [first, last] > > ... > >> However, this is infested with for loops. Therefore > > ... > >> I don't immediately see what to do about the for loop in edges(), so >> I'll use the traditional cop-out: Removing the last loop is left as an >> exercise... > > In the spirit of the exercise: > > def sekond(x, y): > return y > > def edges(items): # where items is a non-empty iterator > first = next(items) > last = functools.reduce(sekond, items, first) > return [first, last] > > Of course, right? Yeah, reduce() is certainly the cherry on the itertools cake ;) From grant.b.edwards at gmail.com Tue Jun 13 17:26:55 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 13 Jun 2017 21:26:55 +0000 (UTC) Subject: How to store some elements from a list into another References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> <34a1e967-cb36-424c-9b3e-2aa44e36212c@googlegroups.com> Message-ID: On 2017-06-13, Peter Otten <__peter__ at web.de> wrote: >> def edges(items): # where items is a non-empty iterator >> first = next(items) >> last = functools.reduce(sekond, items, first) >> return [first, last] >> >> Of course, right? > > Yeah, reduce() is certainly the cherry on the itertools cake ;) Is the optional initializer the only difference between functools.reduce() and the builtin reduce()? -- Grant Edwards grant.b.edwards Yow! I know th'MAMBO!! at I have a TWO-TONE CHEMISTRY gmail.com SET!! From __peter__ at web.de Tue Jun 13 17:37:05 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 13 Jun 2017 23:37:05 +0200 Subject: How to store some elements from a list into another References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> <34a1e967-cb36-424c-9b3e-2aa44e36212c@googlegroups.com> Message-ID: Grant Edwards wrote: > On 2017-06-13, Peter Otten <__peter__ at web.de> wrote: > >>> def edges(items): # where items is a non-empty iterator >>> first = next(items) >>> last = functools.reduce(sekond, items, first) >>> return [first, last] >>> >>> Of course, right? >> >> Yeah, reduce() is certainly the cherry on the itertools cake ;) > > Is the optional initializer the only difference between > functools.reduce() and the builtin reduce()? I don't think there's a difference at all -- at least the docstrings are the same: $ python -c 'import functools; print functools.reduce.__doc__ == reduce.__doc__; print reduce.__doc__' True reduce(function, sequence[, initial]) -> value Apply a function of two arguments cumulatively to the items of a sequence, from left to right, so as to reduce the sequence to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). If initial is present, it is placed before the items of the sequence in the calculation, and serves as a default when the sequence is empty. $ Note that the builtin was removed in Python 3. From grant.b.edwards at gmail.com Tue Jun 13 18:21:37 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 13 Jun 2017 22:21:37 +0000 (UTC) Subject: How to store some elements from a list into another References: <9d9fa06e-8d4e-4516-9c50-b82ab8bccfb0@googlegroups.com> <34a1e967-cb36-424c-9b3e-2aa44e36212c@googlegroups.com> Message-ID: On 2017-06-13, Peter Otten <__peter__ at web.de> wrote: > Grant Edwards wrote: > >> On 2017-06-13, Peter Otten <__peter__ at web.de> wrote: >> >>>> def edges(items): # where items is a non-empty iterator >>>> first = next(items) >>>> last = functools.reduce(sekond, items, first) >>>> return [first, last] >>>> >>>> Of course, right? >>> >>> Yeah, reduce() is certainly the cherry on the itertools cake ;) >> >> Is the optional initializer the only difference between >> functools.reduce() and the builtin reduce()? > > I don't think there's a difference at all -- at least the docstrings are the > same: > > $ python -c 'import functools; print functools.reduce.__doc__ == > reduce.__doc__; print reduce.__doc__' > True > reduce(function, sequence[, initial]) -> value Hmm. I don't know where I got the impression that the built-in didn't support the optional initializer. It's clearly there in the official docs. I must have been accidentally looking at one of those bogus "tutorial" sites that have managed to fool Google into thinking they're not worthless tripe. > Note that the builtin was removed in Python 3. Yep, I just figured that out. [I still use 2.7 for most of my quick/small applications since being able to freely mix strings and bytes saves a lot of hassle for the work I do.] -- Grant Edwards grant.b.edwards Yow! We just joined the at civil hair patrol! gmail.com From matt.mailinglists at gmail.com Tue Jun 13 18:34:44 2017 From: matt.mailinglists at gmail.com (Matt) Date: Tue, 13 Jun 2017 17:34:44 -0500 Subject: Test String Contents Message-ID: What is easiest way to determine if a string ONLY contains a-z upper or lowercase. I also want to allow the "-" and "_" symbols. No spaces or anything else. From tomuxiong at gmx.com Tue Jun 13 18:49:20 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Tue, 13 Jun 2017 15:49:20 -0700 Subject: Test String Contents In-Reply-To: References: Message-ID: On 06/13/2017 03:34 PM, Matt wrote: > What is easiest way to determine if a string ONLY contains a-z upper > or lowercase. I also want to allow the "-" and "_" symbols. No > spaces or anything else. > I'm not sure it's the best way, but the following seems okay: >>> s = 'hello_world' >>> s.replace('-','').replace('_','').isalpha() True >>> s = 'hello world' >>> s.replace('-','').replace('_','').isalpha() False >>> Cheers, Thomas From __peter__ at web.de Tue Jun 13 18:50:37 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 14 Jun 2017 00:50:37 +0200 Subject: Test String Contents References: Message-ID: Matt wrote: > What is easiest way to determine if a string ONLY contains a-z upper > or lowercase. I also want to allow the "-" and "_" symbols. No > spaces or anything else. If you don't know regular expressions here's a method where not much can go wrong: >>> import string >>> acceptable = frozenset(string.ascii_letters + "_-").issuperset >>> acceptable("Foo-Bar") True >>> acceptable("Foo-Bar42") False >>> acceptable("Foo Bar") False >>> acceptable(string.ascii_letters + "_-") True From __peter__ at web.de Tue Jun 13 21:08:36 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 14 Jun 2017 03:08:36 +0200 Subject: Test String Contents References: Message-ID: Peter Otten wrote: > Matt wrote: > >> What is easiest way to determine if a string ONLY contains a-z upper >> or lowercase. I also want to allow the "-" and "_" symbols. No >> spaces or anything else. > > If you don't know regular expressions here's a method where not much can > go wrong: ... with the exception of the empty string. If you want to reject empty strings you need to add an explicit test: if s and acceptable(s): print("OK") >>>> import string >>>> acceptable = frozenset(string.ascii_letters + "_-").issuperset >>>> acceptable("Foo-Bar") > True >>>> acceptable("Foo-Bar42") > False >>>> acceptable("Foo Bar") > False >>>> acceptable(string.ascii_letters + "_-") > True By the way, I find that behaviour more intuitive than that of the str.isXXX() methods as e. g. s.isdigit() will differ from all(c.isdigit() for c in s) for the empty string. What do you think, are all apples in an empty basket red or not? From steve at pearwood.info Wed Jun 14 00:25:07 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 14 Jun 2017 04:25:07 GMT Subject: ensurepip Message-ID: <5940baa3$0$2894$c3e8da3$76491128@news.astraweb.com> ensurepip is added in Python 3.4: https://docs.python.org/3/library/ensurepip.html But: root at runes:~# python3.4 -m ensurepip /usr/bin/python3.4: No module named ensurepip Any clues on what is going on or how to diagnose this? -- Steve From pavol.lisy at gmail.com Wed Jun 14 01:00:13 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Wed, 14 Jun 2017 07:00:13 +0200 Subject: ensurepip In-Reply-To: <5940baa3$0$2894$c3e8da3$76491128@news.astraweb.com> References: <5940baa3$0$2894$c3e8da3$76491128@news.astraweb.com> Message-ID: $ python3 -m ensurepip /usr/bin/python3: No module named ensurepip But maybe this help to understand: $ python -m ensurepip ensurepip is disabled in Debian/Ubuntu for the system python. Python modules For the system python are usually handled by dpkg and apt-get. apt-get install python- Install the python-pip package to use pip itself. Using pip together with the system python might have unexpected results for any system installed module, so use it on your own risk, or make sure to only use it in virtual environments. On 6/14/17, Steven D'Aprano wrote: > ensurepip is added in Python 3.4: > > https://docs.python.org/3/library/ensurepip.html > > > But: > > root at runes:~# python3.4 -m ensurepip > /usr/bin/python3.4: No module named ensurepip > > > > Any clues on what is going on or how to diagnose this? > > > > > -- > Steve > -- > https://mail.python.org/mailman/listinfo/python-list > From larry.martell at gmail.com Wed Jun 14 08:50:09 2017 From: larry.martell at gmail.com (Larry Martell) Date: Wed, 14 Jun 2017 08:50:09 -0400 Subject: New to Python - Career question In-Reply-To: <87o9twe0wp.fsf@nightsong.com> References: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> <87o9twe0wp.fsf@nightsong.com> Message-ID: On Sat, Jun 10, 2017 at 3:51 AM, Paul Rubin wrote: > Larry Martell writes: >> I can tell they think I am old and they dismiss me right away. > > http://oldgeekjobs.com ? Cool! Thanks! Sharing with all my old nerdy friends. From sami.strat at gmail.com Wed Jun 14 11:02:58 2017 From: sami.strat at gmail.com (SS) Date: Wed, 14 Jun 2017 08:02:58 -0700 (PDT) Subject: python nmap for loop Message-ID: <7cdf0f6d-7097-4b7c-9d3e-d55965032fda@googlegroups.com> I'm trying to make the following code work: import os, sys app=['host1', 'host2', 'host3'] for i in app: os.system('nmap -p 22 -P0 %s | grep open 2>&1 > /dev/null && echo "%s up" I've tried many different iterations of the os.system call, how to make this work? TIA From saxri89 at gmail.com Wed Jun 14 11:19:17 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Wed, 14 Jun 2017 08:19:17 -0700 (PDT) Subject: dont exetute my exe after decompile and change code Message-ID: <5ffb1171-af51-4c75-918f-cfd38b0c8f94@googlegroups.com> i have an .exe file where file compiled by py2exe in my .exe folder i have some .dll files one .exe file and library.zip file and inside this zip i have to many .pyccombile files. i have decompile this files from library.zip using this program and that program create me new file where i can see and change my code. i have open this file where i need and i change my code using python editor and finaly i save as new script code with the some name and extension .pyc with purpose to replace first .pyc. zip again library folder and i try to run .exe prgram but after the changes the program dont exetute. where step i have wrong in my task ?i need with the some way to re-compile again ? From skip.montanaro at gmail.com Wed Jun 14 11:24:11 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 14 Jun 2017 10:24:11 -0500 Subject: python nmap for loop In-Reply-To: <7cdf0f6d-7097-4b7c-9d3e-d55965032fda@googlegroups.com> References: <7cdf0f6d-7097-4b7c-9d3e-d55965032fda@googlegroups.com> Message-ID: > I'm trying to make the following code work: > ... It seems fairly clear that you've posted code which couldn't possibly run (missing a closing quote and right paren). Let me suggest: 1. You copy and paste directly from a Python (or IPython/Jupyter/IDLE) session, including prompts and output. 2. You tell us what you expected to happen, and what actually happened. Skip From python at bdurham.com Wed Jun 14 14:09:43 2017 From: python at bdurham.com (Malcolm Greene) Date: Wed, 14 Jun 2017 14:09:43 -0400 Subject: Standard lib version of something like enumerate() that takes a max count iteration parameter? Message-ID: <1497463783.4113781.1009502144.56C9EA51@webmail.messagingengine.com> Wondering if there's a standard lib version of something like enumerate() that takes a max count value? Use case: When you want to enumerate through an iterable, but want to limit the number of iterations without introducing if-condition-break blocks in code. Something like: for counter, key in enumerate( some_iterable, max_count=10 ): Thank you, Malcolm From __peter__ at web.de Wed Jun 14 14:34:30 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 14 Jun 2017 20:34:30 +0200 Subject: Standard lib version of something like enumerate() that takes a max count iteration parameter? References: <1497463783.4113781.1009502144.56C9EA51@webmail.messagingengine.com> Message-ID: Malcolm Greene wrote: > Wondering if there's a standard lib version of something like > enumerate() that takes a max count value? > Use case: When you want to enumerate through an iterable, but want to > limit the number of iterations without introducing if-condition-break > blocks in code. > Something like: > > for counter, key in enumerate( some_iterable, max_count=10 ): > Usually you limit the iterable before passing it to enumerate(): for index, value in enumerate(some_seq[:max_count]): ... When you have to deal with arbitrary iterables there's itertools.islice(): from itertools import islice for index, value in enumerate(islice(some_iterable, max_count)): ... Of course for index, value in islice(enumerate(some_iterable), max_count): ... is also possible. From jussi.piitulainen at helsinki.fi Wed Jun 14 14:38:19 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Wed, 14 Jun 2017 21:38:19 +0300 Subject: Standard lib version of something like enumerate() that takes a max count iteration parameter? References: <1497463783.4113781.1009502144.56C9EA51@webmail.messagingengine.com> Message-ID: Malcolm Greene writes: > Wondering if there's a standard lib version of something like > enumerate() that takes a max count value? > Use case: When you want to enumerate through an iterable, but want to > limit the number of iterations without introducing if-condition-break > blocks in code. > Something like: > > for counter, key in enumerate( some_iterable, max_count=10 ): > > > Thank you, > Malcolm for counter, key in zip(range(10), some_iterable): From python at bdurham.com Wed Jun 14 14:53:22 2017 From: python at bdurham.com (Malcolm Greene) Date: Wed, 14 Jun 2017 14:53:22 -0400 Subject: Standard lib version of something like enumerate() that takes a max count iteration parameter? In-Reply-To: References: <1497463783.4113781.1009502144.56C9EA51@webmail.messagingengine.com> Message-ID: <1497466402.492321.1009552520.29BE236F@webmail.messagingengine.com> Thank you Peter and Jussi - both your solutions were very helpful! Malcolm From mradul.k.dhakad at gmail.com Wed Jun 14 14:59:33 2017 From: mradul.k.dhakad at gmail.com (mradul dhakad) Date: Wed, 14 Jun 2017 14:59:33 -0400 Subject: Error while Importing Teradata in Python Message-ID: Hi All , I am new to python .I have installed 3.6.1 python on my computer. when i am trying to import teradata i am getting below error message: Traceback (most recent call last) File "C:\Users\mradul_dhakad\AppData\Local\Programs\Python\ Python36\Hello.py", line 1, in import teradata ModuleNot FoundError: No Module named 'teradata' Could any one please let me know how to resolve this error. Thanks, Mradul From gbs.deadeye at gmail.com Wed Jun 14 15:28:50 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Wed, 14 Jun 2017 19:28:50 +0000 Subject: Standard lib version of something like enumerate() that takes a max count iteration parameter? In-Reply-To: <1497466402.492321.1009552520.29BE236F@webmail.messagingengine.com> References: <1497463783.4113781.1009502144.56C9EA51@webmail.messagingengine.com> <1497466402.492321.1009552520.29BE236F@webmail.messagingengine.com> Message-ID: I'm a fan of infinite sequences. Try out itertools.islice. You should not underestimate this very important module. Please read also the documentation: https://docs.python.org/3.6/library/itertools.html from itertools import islice iterable = range(10000000000) # since Python 3 range is a lazy evaluated object # using this just as a dummy # if you're using legacy Python (2.x), then use the xrange function for it # or you'll get a memory error max_count = 10 step = 1 for i, element in enumerate(islice(iterable, 0, max_count, step), start=1): print(i, element) Greetings Andre From djnight538 at gmail.com Wed Jun 14 16:16:50 2017 From: djnight538 at gmail.com (djnight538 at gmail.com) Date: Wed, 14 Jun 2017 13:16:50 -0700 (PDT) Subject: Ciphers in SSL library python. Message-ID: Hey, I want to use RC4-SHA in python, but when I try to use it, it doesn't get used (If I do cipher() it says none(and handshake fails too)), I've tried to modify the SSL library, but it didn't help at all(Maybe I did something wrong, any help will be appreciated). Is there a way to use the RC4-SHA cipher(0x05). Please, I need help, I've been looking for an answer for days. From rgacote at appropriatesolutions.com Wed Jun 14 16:25:26 2017 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Wed, 14 Jun 2017 16:25:26 -0400 Subject: Ciphers in SSL library python. In-Reply-To: References: Message-ID: 1: Are you 100% sure the server to which you are trying to connect supports RC4-SHA? 2: If you have access to the server, turn on SSH debug mode to watch your client try and connect. I find that to be helpful in debugging many connection issues. On Wed, Jun 14, 2017 at 4:16 PM, wrote: > Hey, I want to use RC4-SHA in python, but when I try to use it, it doesn't > get used (If I do cipher() it says none(and handshake fails too)), I've > tried to modify the SSL library, but it didn't help at all(Maybe I did > something wrong, any help will be appreciated). Is there a way to use the > RC4-SHA cipher(0x05). Please, I need help, I've been looking for an answer > for days. > -- > https://mail.python.org/mailman/listinfo/python-list > -- Raymond Cote, President voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: ray.cote Schedule a meeting: https://calendly.com/ray_cote/60min/ From brad at midwestaftermarket.com Wed Jun 14 16:33:28 2017 From: brad at midwestaftermarket.com (Bradley Cooper) Date: Wed, 14 Jun 2017 13:33:28 -0700 (PDT) Subject: API Help Message-ID: I am working with an API and I get a return response in this format. [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] What is the best way to read through the data? From rosuav at gmail.com Wed Jun 14 16:38:06 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 06:38:06 +1000 Subject: API Help In-Reply-To: References: Message-ID: On Thu, Jun 15, 2017 at 6:33 AM, Bradley Cooper wrote: > I am working with an API and I get a return response in this format. > > > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] > > What is the best way to read through the data? That looks like JSON. Check out Python's json module: https://docs.python.org/3/library/json.html ChrisA From djnight538 at gmail.com Wed Jun 14 16:40:20 2017 From: djnight538 at gmail.com (djnight538 at gmail.com) Date: Wed, 14 Jun 2017 13:40:20 -0700 (PDT) Subject: Ciphers in SSL library python. In-Reply-To: References: Message-ID: To Ray Cote: Hey, I'm "the server(I've written using ssl/socket)" and my client is using RC4-SHA, but I can't make the server to use it. I make " ciphers='RC4-SHA' " in the ssl.wrap_socket. Do I need to modify SSL file or something to make it work? From python at lucidity.plus.com Wed Jun 14 16:47:03 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 14 Jun 2017 21:47:03 +0100 Subject: API Help In-Reply-To: References: Message-ID: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> On 14/06/17 21:38, Chris Angelico wrote: > On Thu, Jun 15, 2017 at 6:33 AM, Bradley Cooper > wrote: >> I am working with an API and I get a return response in this format. >> >> >> [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] >> >> What is the best way to read through the data? > > That looks like JSON. If the keys weren't quoted, I'd agree with you. It looks like a REPL representation of a Python structure to me (list of dicts where the values could also be lists of dicts ... What makes it look like JSON to you? (I'm not arguing, I'm asking what I've missed). E. From brad at midwestaftermarket.com Wed Jun 14 16:54:46 2017 From: brad at midwestaftermarket.com (Bradley Cooper) Date: Wed, 14 Jun 2017 13:54:46 -0700 (PDT) Subject: API Help In-Reply-To: References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> Message-ID: <49763d6d-5051-464f-85a9-c57ecd9d5696@googlegroups.com> Yes it is not json, I did try that with no luck. From python at lucidity.plus.com Wed Jun 14 16:59:58 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 14 Jun 2017 21:59:58 +0100 Subject: API Help In-Reply-To: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> Message-ID: On 14/06/17 21:47, Erik wrote: > What makes it look like JSON to you? (I'm not arguing, I'm asking what > I've missed). If I cut-and-paste the OP's structure into the REPL, it replies with a valid Python structure that's equivalent (albeit with single-quotes instead of double-quotes). So I guess I'm only partially arguing. I agree that it therefore looks like it's not REPL output, but perhaps the actual input that the OP needs to deal with (from whatever). Still not exclusively JSON though ;) To the OP: start a Python shell and paste your structure as follows: foo = [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY1US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] Now you can interrogate your structure interactively to see what might be a good way of "reading through it" (you don't say what you actually want to do). What does "foo[0]" show. What does "foo[0]['inventory'] show? What does "foo[0]['inventory'][1] show? What does "foo[0]['inventory'][1]['warehouseCode'] show? It's just lists (arrays) and dicts (key/value pairs). E. From m at funkyhat.org Wed Jun 14 17:16:57 2017 From: m at funkyhat.org (Matt Wheeler) Date: Wed, 14 Jun 2017 21:16:57 +0000 Subject: API Help In-Reply-To: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> Message-ID: On Wed, 14 Jun 2017 at 21:47 Erik wrote: > On 14/06/17 21:38, Chris Angelico wrote: > > On Thu, Jun 15, 2017 at 6:33 AM, Bradley Cooper > > wrote: > >> I am working with an API and I get a return response in this format. > >> > >> > >> > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] > >> > >> What is the best way to read through the data? > > > > That looks like JSON. > Yes > If the keys weren't quoted, I'd agree with you. > ? JSON keys are quoted > It looks like a REPL representation of a Python structure to me (list of > dicts where the values could also be lists of dicts ... > > What makes it look like JSON to you? (I'm not arguing, I'm asking what > I've missed). > On Wed, 14 Jun 2017 at 21:54 Bradley Cooper wrote: > Yes it is not json, I did try that with no luck. What did you try? % python Python 2.7.13 (default, Feb 17 2017, 23:41:27) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import json >>> s = raw_input() [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY1US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] >>> json.loads(s) [{u'inventory': [{u'quantityAvailable': 0.0, u'warehouseCode': u'UT1-US'}, {u'quantityAvailable': 0.0, u'warehouseCode': u'KY1US'}, {u'quantityAvailable': 14.0, u'warehouseCode': u'TX-1-US'}, {u'quantityAvailable': 4.0, u'warehouseCode': u'CA-1-US'}, {u'quantityAvailable': 1.0, u'warehouseCode': u'AB-1-CA'}, {u'quantityAvailable': 0.0, u'warehouseCode': u'WA-1-US'}, {u'quantityAvailable': 0.0, u'warehouseCode': u'PO-1-CA'}], u'itemNumber': u'75-5044'}] Looks like json to me, and to json.loads, which is probably more authoritative :) -- -- Matt Wheeler http://funkyh.at From grant.b.edwards at gmail.com Wed Jun 14 17:18:23 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 14 Jun 2017 21:18:23 +0000 (UTC) Subject: API Help References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> Message-ID: On 2017-06-14, Erik wrote: > On 14/06/17 21:38, Chris Angelico wrote: >> On Thu, Jun 15, 2017 at 6:33 AM, Bradley Cooper >> wrote: >>> I am working with an API and I get a return response in this format. >>> >>> >>> {"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] >>> >>> What is the best way to read through the data? >> >> That looks like JSON. Yep. > If the keys weren't quoted, I'd agree with you. JSON keys _are_ quoted. > It looks like a REPL representation of a Python structure to me (list of > dicts where the values could also be lists of dicts ... > > What makes it look like JSON to you? The fact that it _is_ valid JSON (at least according to the parsers I've tried on it, both locally and using things like jsonlint.com). > (I'm not arguing, I'm asking what I've missed). -- Grant Edwards grant.b.edwards Yow! I request a weekend in at Havana with Phil Silvers! gmail.com From rosuav at gmail.com Wed Jun 14 17:30:30 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 07:30:30 +1000 Subject: API Help In-Reply-To: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> Message-ID: On Thu, Jun 15, 2017 at 6:47 AM, Erik wrote: > On 14/06/17 21:38, Chris Angelico wrote: >> >> On Thu, Jun 15, 2017 at 6:33 AM, Bradley Cooper >> wrote: >>> >>> I am working with an API and I get a return response in this format. >>> >>> >>> >>> [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] >>> >>> What is the best way to read through the data? >> >> >> That looks like JSON. > > > If the keys weren't quoted, I'd agree with you. > > It looks like a REPL representation of a Python structure to me (list of > dicts where the values could also be lists of dicts ... > > What makes it look like JSON to you? (I'm not arguing, I'm asking what I've > missed). JSON, unlike JavaScript/ECMAScript itself, requires the keys to be quoted. (It's legal to quote them in JS, but conventionally, people omit the quotes where possible.) It wouldn't be the repr() of a Python structure, as that wouldn't have all those trailing zeroes. But you probably could parse it with literal_eval all the same. Still, I'd go with JSON if at all possible. ChrisA From python at lucidity.plus.com Wed Jun 14 17:33:39 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 14 Jun 2017 22:33:39 +0100 Subject: API Help In-Reply-To: References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> Message-ID: <7ecec35e-15e5-eeb6-2162-51be68b19144@lucidity.plus.com> On 14/06/17 22:18, Grant Edwards wrote: >> What makes it look like JSON to you? > > The fact that it _is_ valid JSON (at least according to the parsers > I've tried on it, both locally and using things like jsonlint.com). And I tried it on the Python REPL. It's Python too. If someone wrote the following on this list: foo = bar(1, x, 3 * (x - 1), frob("hello world")) .. and I said "That looks like C to me", I suspect you and a lot of other people would ask me what makes me think that is C and not something else. Like Python, for example. >> (I'm not arguing, I'm asking what I've missed). My point was (I thought clearly) "What makes this exclusively JSON and not something else"? I even suggested that I was possibly missing something so as to come across as questioning and not argumentative. I guess that didn't work either. E. From python at lucidity.plus.com Wed Jun 14 17:38:39 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 14 Jun 2017 22:38:39 +0100 Subject: API Help In-Reply-To: References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> Message-ID: <6abc610e-a06a-5977-3e09-acec4b6fe9f6@lucidity.plus.com> On 14/06/17 22:16, Matt Wheeler wrote: > ? JSON keys are quoted Thanks Matt, I was confusing myself between JS source and JSON. Good to have this reminder (I always use libraries for reading and writing JSON in whatever language, so while I view it often I very rarely have to type it in directly). E. From __peter__ at web.de Wed Jun 14 17:43:13 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 14 Jun 2017 23:43:13 +0200 Subject: API Help References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> <49763d6d-5051-464f-85a9-c57ecd9d5696@googlegroups.com> Message-ID: Bradley Cooper wrote: > Yes it is not json, I did try that with no luck. What exactly did you try and how did it fail? From rgacote at appropriatesolutions.com Wed Jun 14 17:48:55 2017 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Wed, 14 Jun 2017 17:48:55 -0400 Subject: Ciphers in SSL library python. In-Reply-To: References: Message-ID: On Wed, Jun 14, 2017 at 4:40 PM, wrote: > Hey, I'm "the server(I've written using ssl/socket)" and my client is > using RC4-SHA, but I can't make the server to use it. I make " > ciphers='RC4-SHA' " in the ssl.wrap_socket. Do I need to modify SSL file or > something to make it work? Had not realized you were the server. Been a long time since I used SSL, but here?s a few thoughts. 1: What version of Python are you using? 2: What version of OpenSSL are you using? RC4-SHA is considered insecure and might be disabled at the OpenSSL/OS level. 3: You say you set ciphers=. I found some old code that used the set_ciphers context call. That might be the approach. ?R -- Raymond Cote, President voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: ray.cote Schedule a meeting: https://calendly.com/ray_cote/60min/ From rgacote at appropriatesolutions.com Wed Jun 14 17:54:31 2017 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Wed, 14 Jun 2017 17:54:31 -0400 Subject: API Help In-Reply-To: References: Message-ID: On Wed, Jun 14, 2017 at 4:33 PM, Bradley Cooper wrote: > I am working with an API and I get a return response in this format. > > > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":" > UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US"," > quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US"," > quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US"," > quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA"," > quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US"," > quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA"," > quantityAvailable":0.0000000000000}]}] > > What is the best way to read through the data? > Definitely JSON: >>> json.loads(?""[{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}]""") [{u?itemNumber': u'75-5044', u'inventory': [{u'quantityAvailable': 0.0, u'warehouseCode': u'UT-1-US'}, {u'quantityAvailable': 0.0, u'warehouseCode': u'KY-1-US'}, {u'quantityAvailable': 14.0, u'warehouseCode': u'TX-1-US'}, {u'quantityAvailable': 4.0, u'warehouseCode': u'CA-1-US'}, {u'quantityAvailable': 1.0, u'warehouseCode': u'AB-1-CA'}, {u'quantityAvailable': 0.0, u'warehouseCode': u'WA-1-US'}, {u'quantityAvailable': 0.0, u'warehouseCode': u'PO-1-CA'}]}] >>> -- Raymond Cote, President voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: ray.cote Schedule a meeting: https://calendly.com/ray_cote/60min/ From gbs.deadeye at gmail.com Wed Jun 14 18:10:31 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=c3=bcller?=) Date: Thu, 15 Jun 2017 00:10:31 +0200 Subject: API Help In-Reply-To: References: Message-ID: <4e0393c7-3142-3bd0-dfb0-ce9f32d7a612@gmail.com> Am 14.06.2017 um 22:33 schrieb Bradley Cooper: > I am working with an API and I get a return response in this format. > > > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] > > What is the best way to read through the data? > Your data looks like Json. First use the _json_ module to convert the _string_ into a _Python data structure_. Then you can iterate over it. My example is a nested loop. Maybe you've more than one element inside the list. import json # the string looks like json JSON_DATA = """[ {"itemNumber":"75-5044","inventory": [ {"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000}, {"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000}, {"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000}, {"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000}, {"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000}, {"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000}, {"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000} ] } ]""" # converting the json string to a Python data structure inventory = json.loads(JSON_DATA) # representation in Python #[{'inventory': [{'quantityAvailable': 0.0, 'warehouseCode': 'UT-1-US'}, #{'quantityAvailable': 0.0, 'warehouseCode': 'KY-1-US'}, #{'quantityAvailable': 14.0, 'warehouseCode': 'TX-1-US'}, #{'quantityAvailable': 4.0, 'warehouseCode': 'CA-1-US'}, #{'quantityAvailable': 1.0, 'warehouseCode': 'AB-1-CA'}, #{'quantityAvailable': 0.0, 'warehouseCode': 'WA-1-US'}, #{'quantityAvailable': 0.0, 'warehouseCode': 'PO-1-CA'}], #'itemNumber': '75-5044'}] # the interesting part for items in inventory: # get the elements in from the list # the elements are dicts, in this case exactly one dict print('itemNumber:', i['itemNumber']) # nested loop iterating over the values of the key 'inventory' for item in items['inventory']: # the value is dict code, qty = item['warehouseCode'],item['quantityAvailable'] print('warehouseCode:', code, 'quantityAvailable', qty) # Output #itemNumber: 75-5044 #warehouseCode: UT-1-US quantityAvailable 0.0 #warehouseCode: KY-1-US quantityAvailable 0.0 #warehouseCode: TX-1-US quantityAvailable 14.0 #warehouseCode: CA-1-US quantityAvailable 4.0 #warehouseCode: AB-1-CA quantityAvailable 1.0 #warehouseCode: WA-1-US quantityAvailable 0.0 #warehouseCode: PO-1-CA quantityAvailable 0.0 Greetings Andre -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 866 bytes Desc: OpenPGP digital signature URL: From python at lucidity.plus.com Wed Jun 14 18:14:46 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 14 Jun 2017 23:14:46 +0100 Subject: API Help In-Reply-To: References: Message-ID: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> On 14/06/17 22:54, Ray Cote wrote: > Definitely JSON: >>>> > json.loads(?""[{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}]""") > > [{u?itemNumber': u'75-5044', u'inventory': [{u'quantityAvailable': 0.0, > u'warehouseCode': u'UT-1-US'}, {u'quantityAvailable': 0.0, > u'warehouseCode': u'KY-1-US'}, {u'quantityAvailable': 14.0, > u'warehouseCode': u'TX-1-US'}, {u'quantityAvailable': 4.0, > u'warehouseCode': u'CA-1-US'}, {u'quantityAvailable': 1.0, > u'warehouseCode': u'AB-1-CA'}, {u'quantityAvailable': 0.0, > u'warehouseCode': u'WA-1-US'}, {u'quantityAvailable': 0.0, > u'warehouseCode': u'PO-1-CA'}]}] >>>> If that makes it definitely JSON, then this makes it definitely Python ;) : >>> [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] [{'itemNumber': '75-5044', 'inventory': [{'quantityAvailable': 0.0, 'warehouseCode': 'UT-1-US'}, {'quantityAvailable': 0.0, 'warehouseCode': 'KY-1-US'}, {'quantityAvailable': 14.0, 'warehouseCode': 'TX-1-US'}, {'quantityAvailable': 4.0, 'warehouseCode': 'CA-1-US'}, {'quantityAvailable': 1.0, 'warehouseCode': 'AB-1-CA'}, {'quantityAvailable': 0.0, 'warehouseCode': 'WA-1-US'}, {'quantityAvailable': 0.0, 'warehouseCode': 'PO-1-CA'}]}] So, I think we're agreed that it's definitely one or the other. E. From python at lucidity.plus.com Wed Jun 14 18:34:37 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 14 Jun 2017 23:34:37 +0100 Subject: API Help In-Reply-To: References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> Message-ID: <53beeec5-6ff1-7ab2-2290-1f8604522246@lucidity.plus.com> On 14/06/17 22:30, Chris Angelico wrote: > It wouldn't be the repr() of a Python structure, as that wouldn't have > all those trailing zeroes. That depends on what class represents those float values. It doesn't have to be the built-in float. Same with the double-quotes instead of single-quotes on the strings. I still think it _could_ be the output of a Python repr() or similar (something that is expected to be evaluated as a Python expression). I'm not saying it *is*, and secretly I agree that it's probably not, but it _could_ be. E. From grant.b.edwards at gmail.com Wed Jun 14 18:49:36 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 14 Jun 2017 22:49:36 +0000 (UTC) Subject: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On 2017-06-14, Erik wrote: > On 14/06/17 22:54, Ray Cote wrote: >> Definitely JSON: >>>>> >> json.loads(?""[{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}]""") [...] > > If that makes it definitely JSON, then this makes it definitely Python ;) : > >>> > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] Indeed. > So, I think we're agreed that it's definitely one or the other. It's both a floor wax _and_ a dessert topping! -- Grant From rgaddi at highlandtechnology.invalid Wed Jun 14 19:05:07 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 14 Jun 2017 16:05:07 -0700 Subject: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On 06/14/2017 03:49 PM, Grant Edwards wrote: > On 2017-06-14, Erik wrote: >> On 14/06/17 22:54, Ray Cote wrote: >>> Definitely JSON: >>>>>> >>> json.loads(?""[{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}]""") > [...] >> >> If that makes it definitely JSON, then this makes it definitely Python ;) : >>>>> >> [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] > > Indeed. > >> So, I think we're agreed that it's definitely one or the other. > > It's both a floor wax _and_ a dessert topping! > Since it's viable directly as Python code, you should just eval() the random thing you got from the Internet and use the result.[*]_ .. [*] Don't do that. Don't ever, EVER do that. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From walters.justin01 at gmail.com Wed Jun 14 19:06:20 2017 From: walters.justin01 at gmail.com (justin walters) Date: Wed, 14 Jun 2017 16:06:20 -0700 Subject: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Wed, Jun 14, 2017 at 3:49 PM, Grant Edwards wrote: > On 2017-06-14, Erik wrote: > > On 14/06/17 22:54, Ray Cote wrote: > >> Definitely JSON: > >>>>> > >> json.loads(?""[{"itemNumber":"75-5044","inventory":[{" > warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{" > warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{" > warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{" > warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{" > warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{" > warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{" > warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}]""") > [...] > > > > If that makes it definitely JSON, then this makes it definitely Python > ;) : > > >>> > > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":" > UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US"," > quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US"," > quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US"," > quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA"," > quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US"," > quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA"," > quantityAvailable":0.0000000000000}]}] > > Indeed. > > > So, I think we're agreed that it's definitely one or the other. > > It's both a floor wax _and_ a dessert topping! > > -- > Grant > > -- > https://mail.python.org/mailman/listinfo/python-list > JSON and Python dictionaries have nearly the exact same syntax. That's why working with JSON in Python is such a joy! :) You can paste any valid JSON into a Python REPL and it will be interpreted as a Python dictionary. I can assure you with 99.9999~% confidence that the data you are receiving from the API is JSON data. You will find very few web APIs around nowadays that aren't using JSON or JSONB(binary representation of JSON). All that said, you can use the built in ``json`` module to convert the raw data into a Python dictionary in the following manner(assuming you are using Python 3.2+). ``` import json response_data = "..." # This is the received JSON data as a binary string. data = json.loads( str(response_data, encoding="utf8") ) data["itemNumber"] >>> "75-5044" ``` Hope that helps. From walters.justin01 at gmail.com Wed Jun 14 19:08:26 2017 From: walters.justin01 at gmail.com (justin walters) Date: Wed, 14 Jun 2017 16:08:26 -0700 Subject: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Wed, Jun 14, 2017 at 4:06 PM, justin walters wrote: > > > On Wed, Jun 14, 2017 at 3:49 PM, Grant Edwards > wrote: > >> On 2017-06-14, Erik wrote: >> > On 14/06/17 22:54, Ray Cote wrote: >> >> Definitely JSON: >> >>>>> >> >> json.loads(?""[{"itemNumber":"75-5044","inventory":[{"wareho >> useCode":"UT-1-US","quantityAvailable":0.0000000000000},{"wa >> rehouseCode":"KY-1-US","quantityAvailable":0.0000000000000}, >> {"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000 >> 000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000 >> 000000000},{"warehouseCode":"AB-1-CA","quantityAvailable": >> 1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailab >> le":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAva >> ilable":0.0000000000000}]}]""") >> [...] >> > >> > If that makes it definitely JSON, then this makes it definitely Python >> ;) : >> > >>> >> > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT- >> 1-US","quantityAvailable":0.0000000000000},{"warehouseCode >> ":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouse >> Code":"TX-1-US","quantityAvailable":14.0000000000000},{"ware >> houseCode":"CA-1-US","quantityAvailable":4.0000000000000},{" >> warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000 >> },{"warehouseCode":"WA-1-US","quantityAvailable":0.000000000 >> 0000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] >> >> Indeed. >> >> > So, I think we're agreed that it's definitely one or the other. >> >> It's both a floor wax _and_ a dessert topping! >> >> -- >> Grant >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > JSON and Python dictionaries have nearly the exact same syntax. That's why > working with > JSON in Python is such a joy! :) > > You can paste any valid JSON into a Python REPL and it will be interpreted > as a Python > dictionary. > > I can assure you with 99.9999~% confidence that the data you are receiving > from the API > is JSON data. You will find very few web APIs around nowadays that aren't > using JSON or > JSONB(binary representation of JSON). > > All that said, you can use the built in ``json`` module to convert the raw > data into a Python > dictionary in the following manner(assuming you are using Python 3.2+). > > ``` > import json > > response_data = "..." # This is the received JSON data as a binary string. > > data = json.loads( > str(response_data, encoding="utf8") > ) > > data["itemNumber"] > > >>> "75-5044" > ``` > > Hope that helps. > I should also specify that the data you have received is malformed. JSON should always have an object as the top level structure, not an array as in your data. From python at lucidity.plus.com Wed Jun 14 19:17:41 2017 From: python at lucidity.plus.com (Erik) Date: Thu, 15 Jun 2017 00:17:41 +0100 Subject: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: <44103ae6-5942-02a7-b0be-9093a4dd2894@lucidity.plus.com> On 15/06/17 00:08, justin walters wrote: > I should also specify that the data you have received is malformed. > > JSON should always have an object as the top level structure, not an > array as in your data. So it's not JSON then? :D E. From grant.b.edwards at gmail.com Wed Jun 14 19:40:58 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 14 Jun 2017 23:40:58 +0000 (UTC) Subject: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On 2017-06-14, justin walters wrote: > I should also specify that the data you have received is malformed. > > JSON should always have an object as the top level structure, not an > array as in your data. Where is that requirement stated? RFC7159 explicitly states that a "conforming JSON text" can be either an array or an object: 2. JSON Grammar A JSON text is a sequence of tokens. The set of tokens includes six structural characters, strings, numbers, and three literal names. A JSON text is a serialized value. Note that certain previous specifications of JSON constrained a JSON text to be an object or an array. Implementations that generate only objects or arrays where a JSON text is called for will be interoperable in the sense that all implementations will accept these as conforming JSON texts. The grammar specified by later in that section also allow a JSON text to contain nothing but a single simple value: string, number 'null' 'true' or 'false'. IOW, a file containing a single digit (0-9) is valid JSON. -- Grant From rosuav at gmail.com Wed Jun 14 19:45:08 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 09:45:08 +1000 Subject: API Help In-Reply-To: <53beeec5-6ff1-7ab2-2290-1f8604522246@lucidity.plus.com> References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> <53beeec5-6ff1-7ab2-2290-1f8604522246@lucidity.plus.com> Message-ID: On Thu, Jun 15, 2017 at 8:34 AM, Erik wrote: > On 14/06/17 22:30, Chris Angelico wrote: >> >> It wouldn't be the repr() of a Python structure, as that wouldn't have >> all those trailing zeroes. > > > That depends on what class represents those float values. It doesn't have to > be the built-in float. Same with the double-quotes instead of single-quotes > on the strings. > > I still think it _could_ be the output of a Python repr() or similar > (something that is expected to be evaluated as a Python expression). I'm not > saying it *is*, and secretly I agree that it's probably not, but it _could_ > be. JSON mandates double quotes, even though JS and Python both support single. Python's built-in float won't repr like that. Add all that kind of thing together, and you get my original conclusion that this is JSON. It's also perfectly parseable as various other things, but it's still most likely to be JSON. But... On Thu, Jun 15, 2017 at 9:08 AM, justin walters wrote: > I should also specify that the data you have received is malformed. > > JSON should always have an object as the top level structure, not an > array as in your data. ... JSON doesn't actually mandate that. You can use any top-level type. The string '"hello"' is a valid JSON-decodable value. ChrisA From amirouche.boubekki at gmail.com Wed Jun 14 19:46:39 2017 From: amirouche.boubekki at gmail.com (Amirouche Boubekki) Date: Wed, 14 Jun 2017 23:46:39 +0000 Subject: How do you use Python 3.5 and Python 3.6 in production Message-ID: H?llo, I'd like to use Python 3.5 or Python 3.6 in production but avoid the use of pip and virtualenv. Is there a solution based on a popular GNU/Linux distribution that allows to keep up with the release of Python 3.x and various other highly prolific project but still young like aiohttp? What I am looking for is the ability to have reproducible builds in the form of lxc templates (or maybe somekind of docker magic) but without the need to maintain another package repository. I hope my question is clear. Thanks in advance! From python at lucidity.plus.com Wed Jun 14 20:07:04 2017 From: python at lucidity.plus.com (Erik) Date: Thu, 15 Jun 2017 01:07:04 +0100 Subject: API Help In-Reply-To: References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> <53beeec5-6ff1-7ab2-2290-1f8604522246@lucidity.plus.com> Message-ID: <7434a740-30aa-9ded-acf4-a944c5e313af@lucidity.plus.com> On 15/06/17 00:45, Chris Angelico wrote: > Add all that > kind of thing together, and you get my original conclusion that this > is JSON. It's also perfectly parseable as various other things, but > it's still most likely to be JSON. It "is JSON" but is also "parsable as various other things" yet is "most likely to be JSON". I think it is reasonable to interpret that as "Might not be JSON", which was my original point (ignoring my fubar with the quoted keys thing). E. From gbs.deadeye at gmail.com Wed Jun 14 20:08:51 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Thu, 15 Jun 2017 00:08:51 +0000 Subject: Converting epoch to string in format yyyy-mm-dd, or maybe it is not necessary In-Reply-To: <87tw3ylxm1.fsf@handshake.de> References: <92bbcf4a-07ec-4184-a8ed-5fbeb5e58d92@googlegroups.com> <87tw3ylxm1.fsf@handshake.de> Message-ID: I'm not familar with pandas. If you look on stackoverfolow you'll find this solution: df.epoch = pd.to_datetime(df.epoch) https://stackoverflow.com/questions/17134716/convert-dataframe-column-type-from-string-to-datetime But in this case, it's not a plain string, then it's a datetime object. Greetings Andre From gbs.deadeye at gmail.com Wed Jun 14 20:20:38 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Thu, 15 Jun 2017 00:20:38 +0000 Subject: How do you use Python 3.5 and Python 3.6 in production In-Reply-To: References: Message-ID: Hi, I'm using Arch Linux. There is currently Python 3.6 the standard interpreter. But I think it's not good to use this in production. Hm, maybe pyenv can be an distribution independent solution: https://github.com/pyenv/pyenv If you're using pyenv, then you'll have some build dependencies. One time I've used a Debian repository to install Python 3.6 somewhere (can not remind where). But then you rely on the user, who is managing the repository. I often build my own Python version. If you're willing to do this, you can build your own packages on your own repository. Then you can install this version on your production without having there the whole build dependencies. But this is additional work. Amirouche Boubekki schrieb am Do., 15. Juni 2017 um 01:47 Uhr: > H?llo, > > > I'd like to use Python 3.5 or Python 3.6 in production but avoid the use of > pip and virtualenv. > > Is there a solution based on a popular GNU/Linux distribution that allows > to keep up with the release of Python 3.x and various other highly prolific > project but still young like aiohttp? > > What I am looking for is the ability to have reproducible builds in the > form of lxc templates (or maybe somekind of docker magic) but without the > need to maintain another package repository. > > I hope my question is clear. > > Thanks in advance! > -- > https://mail.python.org/mailman/listinfo/python-list > From rgacote at appropriatesolutions.com Wed Jun 14 20:30:30 2017 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Wed, 14 Jun 2017 20:30:30 -0400 Subject: API Help In-Reply-To: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Wed, Jun 14, 2017 at 6:14 PM, Erik wrote: > On 14/06/17 22:54, Ray Cote wrote: > >> Definitely JSON: >> >>> >>>>> json.loads(?""[{"itemNumber":"75-5044","inventory":[{"wareho >> useCode":"UT-1-US","quantityAvailable":0.0000000000000},{"wa >> rehouseCode":"KY-1-US","quantityAvailable":0.0000000000000}, >> {"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000 >> 000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000 >> 000000000},{"warehouseCode":"AB-1-CA","quantityAvailable": >> 1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailab >> le":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAva >> ilable":0.0000000000000}]}]""") >> >> [{u?itemNumber': u'75-5044', u'inventory': [{u'quantityAvailable': 0.0, >> u'warehouseCode': u'UT-1-US'}, {u'quantityAvailable': 0.0, >> u'warehouseCode': u'KY-1-US'}, {u'quantityAvailable': 14.0, >> u'warehouseCode': u'TX-1-US'}, {u'quantityAvailable': 4.0, >> u'warehouseCode': u'CA-1-US'}, {u'quantityAvailable': 1.0, >> u'warehouseCode': u'AB-1-CA'}, {u'quantityAvailable': 0.0, >> u'warehouseCode': u'WA-1-US'}, {u'quantityAvailable': 0.0, >> u'warehouseCode': u'PO-1-CA'}]}] >> >>> >>>>> > If that makes it definitely JSON, then this makes it definitely Python ;) : > > >>> [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT- > 1-US","quantityAvailable":0.0000000000000},{"warehouseCode > ":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouse > Code":"TX-1-US","quantityAvailable":14.0000000000000},{"ware > houseCode":"CA-1-US","quantityAvailable":4.0000000000000},{" > warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000 > },{"warehouseCode":"WA-1-US","quantityAvailable":0.000000000 > 0000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] > [{'itemNumber': '75-5044', 'inventory': [{'quantityAvailable': 0.0, > 'warehouseCode': 'UT-1-US'}, {'quantityAvailable': 0.0, 'warehouseCode': > 'KY-1-US'}, {'quantityAvailable': 14.0, 'warehouseCode': 'TX-1-US'}, > {'quantityAvailable': 4.0, 'warehouseCode': 'CA-1-US'}, > {'quantityAvailable': 1.0, 'warehouseCode': 'AB-1-CA'}, > {'quantityAvailable': 0.0, 'warehouseCode': 'WA-1-US'}, > {'quantityAvailable': 0.0, 'warehouseCode': 'PO-1-CA'}]}] > > > > So, I think we're agreed that it's definitely one or the other. > Well, that is the Python output from the json.loads() call; so yes? -- Raymond Cote, President voice: +1.603.924.6079 email: rgacote at AppropriateSolutions.com skype: ray.cote Schedule a meeting: https://calendly.com/ray_cote/60min/ From ned at nedbatchelder.com Wed Jun 14 21:00:26 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 14 Jun 2017 18:00:26 -0700 (PDT) Subject: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Wednesday, June 14, 2017 at 7:06:39 PM UTC-4, justin walters wrote: > JSON and Python dictionaries have nearly the exact same syntax. That's why > working with > JSON in Python is such a joy! :) > > You can paste any valid JSON into a Python REPL and it will be interpreted > as a Python dictionary. Careful: JSON booleans are true and false, not True and False, and the missing value is null, not None. So valid JSON might not be readable as Python. --Ned. From formisc at gmail.com Wed Jun 14 21:36:50 2017 From: formisc at gmail.com (Andrew Zyman) Date: Wed, 14 Jun 2017 21:36:50 -0400 Subject: data structure Message-ID: Hello, i wonder what would be a proper data structure for something with the following characteristics: id - number, obj[a..c] - objects of various classes the idea is to be able to update certain fields of these objects initially getting access to the record by ID something like this ( not working ) ### code start class ClassA(object): a = '' b = '' def __init__(self): a= 'aa' b= 'ab' class ClassB(object): def __init__(self): self.c = 'ba' self.d = 'bb' def main(): obja = ClassA objb = ClassB sets = set(obja, objb) contracts[1] = sets print('Sets ', contracts) # with the logic like ( not working too) if obja.a = 'aa': contracts[1].obja.a = 'ABC' if __name__ == '__main__': main() ### code end appreciate your guidance From rosuav at gmail.com Wed Jun 14 21:37:44 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 11:37:44 +1000 Subject: How do you use Python 3.5 and Python 3.6 in production In-Reply-To: References: Message-ID: On Thu, Jun 15, 2017 at 9:46 AM, Amirouche Boubekki wrote: > I'd like to use Python 3.5 or Python 3.6 in production but avoid the use of > pip and virtualenv. > > Is there a solution based on a popular GNU/Linux distribution that allows > to keep up with the release of Python 3.x and various other highly prolific > project but still young like aiohttp? > > What I am looking for is the ability to have reproducible builds in the > form of lxc templates (or maybe somekind of docker magic) but without the > need to maintain another package repository. > > I hope my question is clear. I generally build the very latest from source control. (Presumably it'll be the same if you get a .tar.gz archive, but I haven't tested it.) Either way, you'll generally do something like this: $ sudo apt build-dep python3 $ ./configure $ make $ sudo make install and that should give you a fully working Python. And here's the thing: for a modern Python, "fully working" includes having pip available. I'm not sure what the advantage is of avoiding pip, but if your idea is to avoid trampling on the system Python, the easiest way is to use the inbuilt venv (rather than the third-party virtualenv) and use that. To keep up with projects like aiohttp, you're either going to need to use pip, or you're going to install them manually. I strongly recommend the former. $ python3 -m venv env $ source env/bin/activate $ pip install aiohttp That's usually the safest way to do things IMO. ChrisA From torriem at gmail.com Wed Jun 14 22:33:55 2017 From: torriem at gmail.com (Michael Torrie) Date: Wed, 14 Jun 2017 20:33:55 -0600 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On 06/14/2017 05:06 PM, justin walters wrote: > JSON in Python is such a joy! :) I understand that in this case the data is coming from a server in a form intended for easy use with Javascript. But other than this type of communication, I don't see any good reason to choose JSON as a data interchange format. To me JSON seems to hold no real benefits over other serialization techniques, including the XML beast. XML may be verbose, but at least XML data can be formally validated and formally transformed and queried, which is certainly not the case with JSON as we can see from this long thread. JSON's mantra seems to be try parsing it and see what happens. Whether it works depends on the parser and any quirks it has (quotes vs single quotes, etc). I don't find JSON any more human readable than XML to be honest, perhaps less so because there's no way to define a formal schema to follow, at least that I'm aware of. From skip.montanaro at gmail.com Wed Jun 14 22:43:04 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 14 Jun 2017 21:43:04 -0500 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Wed, Jun 14, 2017 at 9:33 PM, Michael Torrie wrote: > To me JSON seems to hold no real benefits over other serialization > techniques, including the XML beast. I guess we'll have to agree to disagree. XML is the Devil's spawn, hiding its real intent inside layer after layer of tags. In contrast, JSON is Hello Kitty, all cute and pink, just dictionaries, lists, and scalars. Skip From walters.justin01 at gmail.com Wed Jun 14 22:46:14 2017 From: walters.justin01 at gmail.com (justin walters) Date: Wed, 14 Jun 2017 19:46:14 -0700 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Wed, Jun 14, 2017 at 7:33 PM, Michael Torrie wrote: > On 06/14/2017 05:06 PM, justin walters wrote: > > JSON in Python is such a joy! :) > > I understand that in this case the data is coming from a server in a > form intended for easy use with Javascript. But other than this type of > communication, I don't see any good reason to choose JSON as a data > interchange format. > > To me JSON seems to hold no real benefits over other serialization > techniques, including the XML beast. XML may be verbose, but at least > XML data can be formally validated and formally transformed and queried, > which is certainly not the case with JSON as we can see from this long > thread. JSON's mantra seems to be try parsing it and see what happens. > Whether it works depends on the parser and any quirks it has (quotes vs > single quotes, etc). I don't find JSON any more human readable than XML > to be honest, perhaps less so because there's no way to define a formal > schema to follow, at least that I'm aware of. > > > -- > https://mail.python.org/mailman/listinfo/python-list > There are 2 main issues with XML: 1) It is not secure. Check this out: https://stackoverflow.com/questions/1906927/xml-vulnerabilities#1907500 2) It is large. JSON can express the same amount of information while using much less memory. There are many reasons for this, but the simplest is that JSON formatting requires less characters. Also, there are several formal schemas to follow. The most popular is JSONAPI. JSON is also fundamentally much simpler than XML. There are strings, numbers, arrays, and objects. That's it. It is basically a dumbed down Python dictionary. From walters.justin01 at gmail.com Wed Jun 14 22:49:08 2017 From: walters.justin01 at gmail.com (justin walters) Date: Wed, 14 Jun 2017 19:49:08 -0700 Subject: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Wed, Jun 14, 2017 at 4:40 PM, Grant Edwards wrote: > On 2017-06-14, justin walters wrote: > > > I should also specify that the data you have received is malformed. > > > > JSON should always have an object as the top level structure, not an > > array as in your data. > > Where is that requirement stated? > > RFC7159 explicitly states that a "conforming JSON text" can be either > an array or an object: > > > 2. JSON Grammar > > A JSON text is a sequence of tokens. The set of tokens includes > six structural characters, strings, numbers, and three literal > names. > > A JSON text is a serialized value. Note that certain previous > specifications of JSON constrained a JSON text to be an object or > an array. Implementations that generate only objects or arrays > where a JSON text is called for will be interoperable in the sense > that all implementations will accept these as conforming JSON > texts. > > The grammar specified by later in that section also allow a JSON text > to contain nothing but a single simple value: string, number 'null' > 'true' or 'false'. IOW, a file containing a single digit (0-9) is > valid JSON. > > -- > Grant > > > > > > > > -- > https://mail.python.org/mailman/listinfo/python-list > That's why I said "should", not "required". Generally, it's considered best practice for web APIs. I also did not say that the data is "invalid", I said it is "malformed". From walters.justin01 at gmail.com Wed Jun 14 22:51:45 2017 From: walters.justin01 at gmail.com (justin walters) Date: Wed, 14 Jun 2017 19:51:45 -0700 Subject: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Wed, Jun 14, 2017 at 6:00 PM, Ned Batchelder wrote: > On Wednesday, June 14, 2017 at 7:06:39 PM UTC-4, justin walters wrote: > > JSON and Python dictionaries have nearly the exact same syntax. That's > why > > working with > > JSON in Python is such a joy! :) > > > > You can paste any valid JSON into a Python REPL and it will be > interpreted > > as a Python dictionary. > > Careful: JSON booleans are true and false, not True and False, and the > missing > value is null, not None. So valid JSON might not be readable as Python. > > --Ned. > -- > https://mail.python.org/mailman/listinfo/python-list > Oh! Yep! I haven't run into this issue because I generally don't run ``eval()`` on random data from the internet. I usually use either the built in ``json`` module or ``marshmallow`` if I have a strict schema to conform to. From grant.b.edwards at gmail.com Wed Jun 14 23:06:40 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 15 Jun 2017 03:06:40 +0000 (UTC) Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On 2017-06-15, Michael Torrie wrote: > On 06/14/2017 05:06 PM, justin walters wrote: >> JSON in Python is such a joy! :) 100% agreement here. > I understand that in this case the data is coming from a server in a > form intended for easy use with Javascript. But other than this > type of communication, I don't see any good reason to choose JSON as > a data interchange format. > > To me JSON seems to hold no real benefits over other serialization > techniques, A JSON library is usually several orders of magnitude smaller, faster, and easier to use than an XML library. To some of us, that matters a lot. > including the XML beast. XML may be verbose, but at least XML data > can be formally validated To paraphrase Knuth: You can prove an XML file is correct -- but it still won't work. ;) Every time I've used anything involving XML, it was a complete and utter nightmare with no two toolsets/libraries able to agree on anything. OTOH, I've been using JSON for years, and never run into problems like that. Verifying files from schemas never worked, transforming files never worked. I hate XML with a passion... JSON does have it's warts: it should allow trailing commas in sequences of object or array entries. Most parsers don't try to do any sort of error recovery: either the file parses, and you get all the data, or it fails and you get nothing. -- Grant From pavol.lisy at gmail.com Wed Jun 14 23:58:50 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Thu, 15 Jun 2017 05:58:50 +0200 Subject: Converting epoch to string in format yyyy-mm-dd, or maybe it is not necessary In-Reply-To: References: <92bbcf4a-07ec-4184-a8ed-5fbeb5e58d92@googlegroups.com> <87tw3ylxm1.fsf@handshake.de> Message-ID: (I am not very familiar with panda too!) In case of his data he needs to set unit to 's' df['dt'] = pd.to_datetime(df.epoch,unit='s') It return utc time from epoch, so maybe this is what he could use -> df['dt'] = df.epoch.apply(time.ctime) or something like (if he needs strings) -> df['dt'] = df.epoch.apply(lambda a:time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(a)) df['dt'] = df.epoch.apply(lambda a:time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(a)) # if utc is desired On 6/15/17, Andre M?ller wrote: > I'm not familar with pandas. > > If you look on stackoverfolow you'll find this solution: > > df.epoch = pd.to_datetime(df.epoch) > https://stackoverflow.com/questions/17134716/convert-dataframe-column-type-from-string-to-datetime > > But in this case, it's not a plain string, then it's a datetime object. > > Greetings > Andre > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Thu Jun 15 00:21:49 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 14:21:49 +1000 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: On Thu, Jun 15, 2017 at 12:33 PM, Michael Torrie wrote: > To me JSON seems to hold no real benefits over other serialization > techniques, including the XML beast. XML may be verbose, but at least > XML data can be formally validated and formally transformed and queried, > which is certainly not the case with JSON as we can see from this long > thread. JSON's mantra seems to be try parsing it and see what happens. > Whether it works depends on the parser and any quirks it has (quotes vs > single quotes, etc). I don't find JSON any more human readable than XML > to be honest, perhaps less so because there's no way to define a formal > schema to follow, at least that I'm aware of. There are JSON schema validators. And I'd trust them every bit as much as I'd trust an XML schema validator - probably more, because I can actually understand them (due largely to the simplicity of JSON). XML is a document format, like HTML and friends. It has certain features that JSON doesn't have, and JSON has features that XML doesn't. Thing is, most people's data is either strictly tabular, or is structured in something like a Python object hierarchy - not a document. XML is thus poorly suited to *most* forms of data, and you'll end up shoehorning data into format and vice versa, which leads to stupidities like being unable to adequately represent a one-item list/array, or having to try to maintain element order. Consider: aa qq qw qe as What does this represent? A generic XML parser has to cope with it. I gave this to a few XML-to-JSON converters, and they all interpreted it as some variant of {"asdf":["aa","as"],"qwer":["qq","qw","qe"]} - but if you remove the last element, they all saw it as {"asdf":"aa","qwer":["qq","qw","qe"]}. So we lose element order (which is a feature of XML but not JSON), and we lose single-element arrays (which are a feature of JSON but not XML), and we make a total hash of the notion of a tree (since both formats are trees but in slightly different ways). Pick a transport format that matches your data structure, or at very least, a strict superset thereof (eg tabular data is probably best expressed as CSV, but JSON is perfectly capable of representing it). XML just needs to die. ChrisA From steve at pearwood.info Thu Jun 15 00:55:55 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 15 Jun 2017 04:55:55 GMT Subject: ensurepip References: <5940baa3$0$2894$c3e8da3$76491128@news.astraweb.com> Message-ID: <5942135b$0$1619$c3e8da3$5496439d@news.astraweb.com> On Wed, 14 Jun 2017 07:00:13 +0200, Pavol Lisy wrote: > $ python3 -m ensurepip /usr/bin/python3: No module named ensurepip > > But maybe this help to understand: > > $ python -m ensurepip > ensurepip is disabled in Debian/Ubuntu for the system python. Ah, thanks! I don't get that error message, possibly because I'm running Python 3 which isn't the system python but was installed by apt-get. -- Steve From jussi.piitulainen at helsinki.fi Thu Jun 15 01:09:58 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 15 Jun 2017 08:09:58 +0300 Subject: Standard lib version of something like enumerate() that takes a max count iteration parameter? References: <1497463783.4113781.1009502144.56C9EA51@webmail.messagingengine.com> <1497466402.492321.1009552520.29BE236F@webmail.messagingengine.com> Message-ID: Andre M?ller writes: > I'm a fan of infinite sequences. Try out itertools.islice. > You should not underestimate this very important module. > > Please read also the documentation: > https://docs.python.org/3.6/library/itertools.html > > from itertools import islice > > iterable = range(10000000000) > # since Python 3 range is a lazy evaluated object > # using this just as a dummy > # if you're using legacy Python (2.x), then use the xrange function for it > # or you'll get a memory error > > max_count = 10 > step = 1 > > for i, element in enumerate(islice(iterable, 0, max_count, step), start=1): > print(i, element) I like to test this kind of thing with iter("abracadabra") and look at the remaining elements, just to be sure that they are still there. from itertools import islice s = iter("abracadabra") for i, element in enumerate(islice(s, 3)): print(i, element) print(''.join(s)) Prints this: 0 a 1 b 2 r acadabra One can do a similar check with iter(range(1000)). The range object itself does not change when its elements are accessed. From marko at pacujo.net Thu Jun 15 01:10:26 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 15 Jun 2017 08:10:26 +0300 Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: <87fuf1def1.fsf@elektro.pacujo.net> Chris Angelico : > XML is thus poorly suited to *most* forms of data, Correct. > > > aa > qq > qw > qe > as > > > What does this represent? A generic XML parser has to cope with it. I > gave this to a few XML-to-JSON converters, and they all interpreted it > as some variant of {"asdf":["aa","as"],"qwer":["qq","qw","qe"]} It's worse than that. For a *generic* parser, it should be something like: { "spam" : [ " \n", { "asdf" : [ "aa" ] }, " \n", { "qwer" : [ "qq" ] }, " \n", { "qwer" : [ "qw" ] }, " \n", { "qwer" : [ "qe" ] }, " \n", { "asdf" : [ "as" ] }, "\n" ] } > XML just needs to die. Thankfully, that seems to be happening. Marko From frank at chagford.com Thu Jun 15 01:28:02 2017 From: frank at chagford.com (Frank Millman) Date: Thu, 15 Jun 2017 07:28:02 +0200 Subject: data structure In-Reply-To: References: Message-ID: "Andrew Zyman" wrote in message news:CAPrcKXKtozoNLaK8ASizoNkYPd9y_P25Fr2rKFkOZxOa4BCMgw at mail.gmail.com... > > Hello, > i wonder what would be a proper data structure for something with the > following characteristics: > > id - number, > obj[a..c] - objects of various classes > > the idea is to be able to update certain fields of these objects initially > getting access to the record by ID > > something like this ( not working ) > > ### code start > > class ClassA(object): > a = '' > b = '' > def __init__(self): > a= 'aa' > b= 'ab' > > class ClassB(object): > def __init__(self): > self.c = 'ba' > self.d = 'bb' > > def main(): > obja = ClassA > objb = ClassB > > sets = set(obja, objb) > contracts[1] = sets > > print('Sets ', contracts) > > # with the logic like ( not working too) > if obja.a = 'aa': > contracts[1].obja.a = 'ABC' > > > if __name__ == '__main__': > main() > > > ### code end I don't really understand the question, but I would like to point out the following - class ClassA(object): a = '' b = '' def __init__(self): a= 'aa' b= 'ab' In your __init__() function, 'a' and 'b' are not bound to anything, so they are simply local variables and will be garbage-collected when the function has completed. Therefore any instance of ClassA() will get its values of 'a' and 'b' from the class definition - an empty string. Frank Millman From greg.ewing at canterbury.ac.nz Thu Jun 15 02:11:25 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 15 Jun 2017 18:11:25 +1200 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: Michael Torrie wrote: > I don't find JSON any more human readable than XML > to be honest, perhaps less so because there's no way to define a formal > schema to follow, at least that I'm aware of. You mean like this? http://json-schema.org/ One thing I like better about JSON than XML as a serialisation format is that JSON constructs map one-to-one with data structures typically found in programming languages, whereas XML constructs don't. With JSON there's usually One Obvious Way to do the mapping, but with XML you get spurious choices and inconvenient restrictions. -- Greg From paul.james.barry at gmail.com Thu Jun 15 03:07:10 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Thu, 15 Jun 2017 08:07:10 +0100 Subject: data structure In-Reply-To: References: Message-ID: Hi Andrew. You start by talking about a data structure, then show code that uses "class". Not everything in Python needs to be in a class. I'd look at using a simple Dictionary of lists, indexed on your ID. A list can contain anything, so you can add your objects in there dynamically as needed. If you need to refer to the objects by name, use a dictionary of dictionaries. Examples: # Dict of lists. >>> my_objects = {} >>> my_objects['id1'] = [] >>> my_objects {'id1': []} >>> my_objects['id2'] = [] >>> my_objects {'id1': [], 'id2': []} >>> my_objects['id3'] = [] >>> my_objects {'id1': [], 'id2': [], 'id3': []} >>> my_objects['id2'].append('some data, but could be anything') >>> my_objects {'id1': [], 'id2': ['some data, but could be anything'], 'id3': []} >>> my_objects['id2'][0] 'some data, but could be anything' # Dict of dicts. >>> my_objects2 = {} >>> my_objects2['id1'] = {} >>> my_objects2['id2'] = {} >>> my_objects2['id3'] = {} >>> my_objects2 {'id1': {}, 'id2': {}, 'id3': {}} >>> my_objects2['id2']['new_key'] = 'some data, but could be anything' >>> my_objects2 {'id1': {}, 'id2': {'new_key': 'some data, but could be anything'}, 'id3': {}} >>> my_objects2['id2']['new_key'] 'some data, but could be anything' I think if you concentrate on manipulating your data as opposed to trying to write code which manipulates it, you might be better off. I hope this helps. Regards. Paul. On 15 June 2017 at 02:36, Andrew Zyman wrote: > Hello, > i wonder what would be a proper data structure for something with the > following characteristics: > > id - number, > obj[a..c] - objects of various classes > > the idea is to be able to update certain fields of these objects initially > getting access to the record by ID > > something like this ( not working ) > > ### code start > > class ClassA(object): > a = '' > b = '' > def __init__(self): > a= 'aa' > b= 'ab' > > class ClassB(object): > def __init__(self): > self.c = 'ba' > self.d = 'bb' > > def main(): > obja = ClassA > objb = ClassB > > sets = set(obja, objb) > contracts[1] = sets > > print('Sets ', contracts) > > # with the logic like ( not working too) > if obja.a = 'aa': > contracts[1].obja.a = 'ABC' > > > if __name__ == '__main__': > main() > > > ### code end > > appreciate your guidance > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From steve at pearwood.info Thu Jun 15 03:14:21 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 15 Jun 2017 07:14:21 GMT Subject: Does Python need Javascript? Message-ID: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> Now that Java 8 includes a Javascript interpreter (Nashorn) as part of the JDK, and since Javascript is The Future?, does Python need a Javascript interpreter in the standard library? If Python came with a Javascript interpreter, what would you do with it? (Serious question, but humorous answers accepted too.) -- Steve From rosuav at gmail.com Thu Jun 15 03:42:12 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 17:42:12 +1000 Subject: Does Python need Javascript? In-Reply-To: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> References: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Jun 15, 2017 at 5:14 PM, Steven D'Aprano wrote: > Now that Java 8 includes a Javascript interpreter (Nashorn) as part of > the JDK, and since Javascript is The Future?, does Python need a > Javascript interpreter in the standard library? > > If Python came with a Javascript interpreter, what would you do with it? > > > > (Serious question, but humorous answers accepted too.) Load up PyPyJS and run Python inside JavaScript inside Python. Just because. ChrisA From python at lucidity.plus.com Thu Jun 15 03:49:29 2017 From: python at lucidity.plus.com (Erik) Date: Thu, 15 Jun 2017 08:49:29 +0100 Subject: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: <5ce9cc82-29cc-3132-6561-af2be9374fbd@lucidity.plus.com> On 15/06/17 01:30, Ray Cote wrote: > On Wed, Jun 14, 2017 at 6:14 PM, Erik > wrote: > If that makes it definitely JSON, then this makes it definitely > Python ;) : > > >>> > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] > [{'itemNumber': '75-5044', 'inventory': [{'quantityAvailable': 0.0, > 'warehouseCode': 'UT-1-US'}, {'quantityAvailable': 0.0, > 'warehouseCode': 'KY-1-US'}, {'quantityAvailable': 14.0, > 'warehouseCode': 'TX-1-US'}, {'quantityAvailable': 4.0, > 'warehouseCode': 'CA-1-US'}, {'quantityAvailable': 1.0, > 'warehouseCode': 'AB-1-CA'}, {'quantityAvailable': 0.0, > 'warehouseCode': 'WA-1-US'}, {'quantityAvailable': 0.0, > 'warehouseCode': 'PO-1-CA'}]}] > > > > So, I think we're agreed that it's definitely one or the other. > > > Well, that is the Python output from the json.loads() call; so yes? No, it is not. It is the output from the Python REPL when the text from the OP is pasted into it (*). My point, if I really have to make it again, is that while the example _could_ be JSON it _could_ equally also be a Python expression. Until the OP says what format it is defined to be or posts an example with something in it that is different between JSON and Python (a boolean, for example) then you can't say it's *definitely* one or the other. E. (*) And to the various people who have responded "I don't eval() random strings from the internet": firstly I eyeballed it to check there was nothing nefarious in it before I pasted it into my interpreter; secondly, I don't think anyone has suggested that is how the OP should actually handle the data. We don't even know if what he posted was the content of a received string or the result of already parsing his data into a structure and printing it out somehow. From paul.james.barry at gmail.com Thu Jun 15 03:50:33 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Thu, 15 Jun 2017 08:50:33 +0100 Subject: Does Python need Javascript? In-Reply-To: References: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: Life is too short... ? On 15 Jun 2017 08:17, "Steven D'Aprano" wrote: Now that Java 8 includes a Javascript interpreter (Nashorn) as part of the JDK, and since Javascript is The Future?, does Python need a Javascript interpreter in the standard library? If Python came with a Javascript interpreter, what would you do with it? (Serious question, but humorous answers accepted too.) -- Steve -- https://mail.python.org/mailman/listinfo/python-list From michele.simionato at gmail.com Thu Jun 15 03:54:13 2017 From: michele.simionato at gmail.com (Michele Simionato) Date: Thu, 15 Jun 2017 00:54:13 -0700 (PDT) Subject: sqlite3 is non-transactional?? Message-ID: <82f8d4da-5c50-435b-ae89-5ee81d9f49cd@googlegroups.com> I know that CREATE queries are non-transactional in sqlite, as documented, but I finding something really strange in INSERT queries. Consider this example: $ cat example.py import os import shutil import sqlite3 script0 = '''\ CREATE TABLE test ( id SERIAL PRIMARY KEY, description TEXT NOT NULL); ''' script1 = '''\ INSERT INTO test (id, description) VALUES (1, 'First'); INSERT INTO test (id, description) VALUES (2, 'Second'); ''' script2 = '''\ INSERT INTO test (id, description) VALUES (1, 'Conflicting with the First'); ''' def main(test_dir): if os.path.exists(test_dir): shutil.rmtree(test_dir) os.mkdir(test_dir) path = os.path.join(test_dir, 'db.sqlite') conn = sqlite3.connect(path) conn.executescript(script0) # this is committing implicitly try: conn.executescript(script1) # this should not be committing conn.executescript(script2) # this one has an error except: conn.rollback() curs = conn.execute('select * from test') for row in curs: # no rows should have been inserted print(row) if __name__ == '__main__': main('/tmp/test') I am creating the test table in script0, populating it in script1, then trying to insert another row with a primary key violation. I would have expected the rollback to remove the rows inserted in script1, since they are part of the same transaction. Instead they are not removed! Can somebody share some light on this? I discover the issue while porting some code from PostgreSQL to sqlite3, with Postgres doing the right thing and sqlite failing. I am puzzled, Michele Simionato From rosuav at gmail.com Thu Jun 15 03:57:21 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 17:57:21 +1000 Subject: Does Python need Javascript? In-Reply-To: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> References: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Jun 15, 2017 at 5:14 PM, Steven D'Aprano wrote: > Now that Java 8 includes a Javascript interpreter (Nashorn) as part of > the JDK, and since Javascript is The Future?, does Python need a > Javascript interpreter in the standard library? > > If Python came with a Javascript interpreter, what would you do with it? Okay, now for a serious response. There've been a number of attempts, over the years, to make a sandboxed Python interpreter, where you can run untrusted code. But if Python came with a JS interpreter, it would be possible to run untrusted JS code, with Python functioning as a gatekeeper. You could allow JS to require("fs"), but still govern which files it reads and writes. You could allow JS to use the fetch() function, but still pick and choose which servers it contacts. It'd also make a great platform for experimenting with ECMAScript itself. Imagine if JS-in-Python had some tweakable parameters that technically violate backward compatibility, but which you can choose to activate from the Python end. You could, for instance: * Make strings truly Unicode, instead of UTF-16 * Introduce an arbitrary-precision integer type * Eliminate the 'var' keyword (just use 'let') * Add a dictionary type that uses non-string keys * Etcetera And possibly most importantly, you could have a high performance data sharing system between the Python and JS sides. A "native code" function could accept any JSON-safe object as a parameter, and directly access its contents (without actually serializing and deserializing). That would allow efficient bilingual code, where anything written in Python can be called from JS and vice versa. Obviously this would depend on the JS objects being internally represented as Python objects, which may have some minor semantic differences; my suspicion is that it'd end up being similar to the way Jython and IronPython behave. But really, I don't think it needs to be in the standard library. A pip-installable third-party implementation would be just as effective. ChrisA From steve at pearwood.info Thu Jun 15 04:17:03 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 15 Jun 2017 08:17:03 GMT Subject: Does Python need Javascript? References: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5942427f$0$1619$c3e8da3$5496439d@news.astraweb.com> On Thu, 15 Jun 2017 17:42:12 +1000, Chris Angelico wrote: > Load up PyPyJS and run Python inside JavaScript inside Python. I'm reminded of a demo I saw once, of a BeOS system running a classic Mac emulator (sheepshaver, I believe it was), then running a Windows emulator inside the Mac emulator. I'd like to say it was still faster than native Windows, but that wouldn't be true. Nevertheless, performance was still reasonable: good enough to run MS Office. -- Steve From __peter__ at web.de Thu Jun 15 04:39:45 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 15 Jun 2017 10:39:45 +0200 Subject: sqlite3 is non-transactional?? References: <82f8d4da-5c50-435b-ae89-5ee81d9f49cd@googlegroups.com> Message-ID: Michele Simionato wrote: > I know that CREATE queries are non-transactional in sqlite, as documented, > but I finding something really strange in INSERT queries. > > Consider this example: > > $ cat example.py > import os > import shutil > import sqlite3 > > script0 = '''\ > CREATE TABLE test ( > id SERIAL PRIMARY KEY, > description TEXT NOT NULL); > ''' > > script1 = '''\ > INSERT INTO test (id, description) > VALUES (1, 'First'); > INSERT INTO test (id, description) > VALUES (2, 'Second'); > ''' > > script2 = '''\ > INSERT INTO test (id, description) > VALUES (1, 'Conflicting with the First'); > ''' > > > def main(test_dir): > if os.path.exists(test_dir): > shutil.rmtree(test_dir) > os.mkdir(test_dir) > path = os.path.join(test_dir, 'db.sqlite') > conn = sqlite3.connect(path) > conn.executescript(script0) # this is committing implicitly > try: > conn.executescript(script1) # this should not be committing > conn.executescript(script2) # this one has an error > except: > conn.rollback() > curs = conn.execute('select * from test') > for row in curs: # no rows should have been inserted > print(row) > > > if __name__ == '__main__': > main('/tmp/test') > > I am creating the test table in script0, populating it in script1, then > trying to insert another row with a primary key violation. I would have > expected the rollback to remove the rows inserted in script1, since they > are part of the same transaction. Instead they are not removed! > > Can somebody share some light on this? I discover the issue while porting > some code from PostgreSQL to sqlite3, with Postgres doing the right thing > and sqlite failing. > > I am puzzled, executescript() is trying to be helpful... """ executescript(sql_script) This is a nonstandard convenience method for executing multiple SQL statements at once. It issues a COMMIT statement first, then executes the SQL script it gets as a parameter. """ ...and failing. When you use execute() things work as expected: $ cat sqlite_trans_demo.py import os import shutil import sqlite3 import sys script0 = '''\ CREATE TABLE test ( id SERIAL PRIMARY KEY, description TEXT NOT NULL); ''' script1 = '''\ INSERT INTO test (id, description) VALUES (1, 'First'); INSERT INTO test (id, description) VALUES (2, 'Second'); ''' script2 = '''\ INSERT INTO test (id, description) VALUES (1, 'Conflicting with the First'); ''' def executescript(conn, script): for sql in script.split(";"): conn.execute(sql) def main(test_dir): if os.path.exists(test_dir): shutil.rmtree(test_dir) os.mkdir(test_dir) path = os.path.join(test_dir, 'db.sqlite') conn = sqlite3.connect(path) conn.executescript(script0) # this is committing implicitly try: executescript(conn, script1) # this should not be committing if "--conflict" in sys.argv: executescript(conn, script2) # this one has an error except Exception as err: print(err) conn.rollback() curs = conn.execute('select * from test') for row in curs: # no rows should have been inserted print(row) if __name__ == '__main__': main('./tmp_sqlite') $ python3 sqlite_trans_demo.py (1, 'First') (2, 'Second') $ python3 sqlite_trans_demo.py --conflict UNIQUE constraint failed: test.id $ From michele.simionato at gmail.com Thu Jun 15 04:50:07 2017 From: michele.simionato at gmail.com (Michele Simionato) Date: Thu, 15 Jun 2017 01:50:07 -0700 (PDT) Subject: sqlite3 is non-transactional?? In-Reply-To: References: <82f8d4da-5c50-435b-ae89-5ee81d9f49cd@googlegroups.com> Message-ID: <0591948c-5a18-4d81-b470-57878b3bde70@googlegroups.com> Thanks. I suspected the culprit was executescript, but I did not see it documented in https://docs.python.org/3/library/sqlite3.html#connection-objects. From m.n.summerfield at googlemail.com Thu Jun 15 05:12:42 2017 From: m.n.summerfield at googlemail.com (Mark Summerfield) Date: Thu, 15 Jun 2017 02:12:42 -0700 (PDT) Subject: sqlite3 is non-transactional?? In-Reply-To: <0591948c-5a18-4d81-b470-57878b3bde70@googlegroups.com> References: <82f8d4da-5c50-435b-ae89-5ee81d9f49cd@googlegroups.com> <0591948c-5a18-4d81-b470-57878b3bde70@googlegroups.com> Message-ID: <6bf4da75-c7e6-4e6a-a65d-b9509399481b@googlegroups.com> On Thursday, June 15, 2017 at 9:50:18 AM UTC+1, Michele Simionato wrote: > Thanks. I suspected the culprit was executescript, but I did not see it documented in https://docs.python.org/3/library/sqlite3.html#connection-objects. Although the standard library's sqlite3 module is useful, personally, I've switched to using APSW which I've found to be extremely reliable and gives much more access to SQLite functionality: https://rogerbinns.github.io/apsw/ From greg.ewing at canterbury.ac.nz Thu Jun 15 05:54:55 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 15 Jun 2017 21:54:55 +1200 Subject: Does Python need Javascript? In-Reply-To: References: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: Chris Angelico wrote: > There've been a number of attempts, over the years, to make a > sandboxed Python interpreter, where you can run untrusted code. But if > Python came with a JS interpreter, it would be possible to run > untrusted JS code, with Python functioning as a gatekeeper. If that would provide sufficient sandboxing, then how about writing a Python interpreter in Python to get a sandboxed Python? -- Greg From djnight538 at gmail.com Thu Jun 15 06:34:34 2017 From: djnight538 at gmail.com (Kacy Night) Date: Thu, 15 Jun 2017 10:34:34 +0000 Subject: Ciphers in SSL Library Message-ID: OS: Windows 7 Home Basic Python: ver. 3.6.0 Hey, I need to use SSLv3 with RC4-SHA (0x05) (Yes I know those are unsecured) but my client is using it and I'm writing a server from scratch in Python. My problem is following, when I try to select cipher it doesn't raise any error, but when client tries to connect with it the handshake fails because server raises error = No Shared Cipher and when I skipped handshake and used at client it said that server had none, but I've selected the RC4-SHA cipher. I've tried to modify the SSL library but it didn't change anything. I'm searching for help with this problem for more than a week. Any help will be appreciated PS. I'm not able to change clients Protocol or Cipher to different. Kacper Z. From pozzugno at gmail.com Thu Jun 15 07:32:10 2017 From: pozzugno at gmail.com (pozz) Date: Thu, 15 Jun 2017 13:32:10 +0200 Subject: [gettext] How to change language at run-time Message-ID: I know I can load multiple gettext.translation: it = gettext.translation('test', localedir="locale", languages=["it"]) es = gettext.translation('test', localedir="locale", languages=["es"]) and install one translation at run-time when I want at a later time (when the user selects a new language): it.install() or es.install() However the problem is that strings already translated are not translated again when a new translation is installed. So they stay at the language selected during start-up and don't change after a new install(). One solution is to restart the application, but I think there's a better and more elegant solution. From rhodri at kynesim.co.uk Thu Jun 15 07:47:31 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 15 Jun 2017 12:47:31 +0100 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> Message-ID: <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> People seem to be having fun bashing XML, so I thought I'd wade in on its behalf. On 15/06/17 03:46, justin walters wrote: > There are 2 main issues with XML: > > 1) It is not secure. Check this out: > https://stackoverflow.com/questions/1906927/xml-vulnerabilities#1907500 XML and JSON share the vulnerabilities that come from having to parse untrusted external input. XML then has some extra since it has extra flexibility, like being able to specify external resources (potential attack vectors) or entity substitution. If you don't need the extra flexibility, feel free to use JSON, but don't for one moment think that makes you inherently safe. > 2) It is large. JSON can express the same amount of information while > using much less memory. There are many reasons for this, but the simplest > is that JSON formatting requires less characters. The simplest, but also the least true. A better reason would be that JSON is a (fairly) simple representation while XML is complex. > Also, there are several formal schemas to follow. The most popular is > JSONAPI. "The nice thing about standards is that you have so many to choose from." > JSON is also fundamentally much simpler than XML. There are strings, > numbers, > arrays, and objects. That's it. It is basically a dumbed down Python > dictionary. In those terms, XML just has elements, attributes and character data, so you don't even have to worry about typing. That's not a useful way of describing JSON's simplicity, though, so it's no wonder XML actually sounds simpler when you put it that way. JSON is a text description of a general purpose data structure. It's relatively simple, relatively easy to parse (which doesn't mean it's easy, I've had some stinkers of pieces of JSON to disentangle in the past), and maps straightforwardly to most people's data transfer needs. XML is a text description of an annotated data structure, with declarative and procedural elements thrown in for extra fun. It's complex, harder to parse, and doesn't have a single obvious mapping to most people's data transfer needs. However when you need that extra flexibility, it's wonderful, and it doesn't *have* to be complex. Of course, all this assumes you don't want the efficiency of a bespoke binary protocol. Living in an embedded world, I usually do :-) -- Rhodri James *-* Kynesim Ltd From brad at midwestaftermarket.com Thu Jun 15 08:04:23 2017 From: brad at midwestaftermarket.com (Bradley Cooper) Date: Thu, 15 Jun 2017 05:04:23 -0700 (PDT) Subject: API Help In-Reply-To: References: <4e0393c7-3142-3bd0-dfb0-ce9f32d7a612@gmail.com> Message-ID: On Wednesday, June 14, 2017 at 6:10:55 PM UTC-4, Andre M?ller wrote: > Am 14.06.2017 um 22:33 schrieb Bradley Cooper: > > I am working with an API and I get a return response in this format. > > > > > > [{"itemNumber":"75-5044","inventory":[{"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000},{"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000},{"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000},{"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000},{"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000}]}] > > > > What is the best way to read through the data? > > > > Your data looks like Json. > First use the _json_ module to convert the _string_ into a _Python data > structure_. > Then you can iterate over it. My example is a nested loop. Maybe > you've more than one element inside the list. > > > import json > > # the string looks like json > JSON_DATA = """[ > {"itemNumber":"75-5044","inventory": [ > {"warehouseCode":"UT-1-US","quantityAvailable":0.0000000000000}, > {"warehouseCode":"KY-1-US","quantityAvailable":0.0000000000000}, > {"warehouseCode":"TX-1-US","quantityAvailable":14.0000000000000}, > {"warehouseCode":"CA-1-US","quantityAvailable":4.0000000000000}, > {"warehouseCode":"AB-1-CA","quantityAvailable":1.0000000000000}, > {"warehouseCode":"WA-1-US","quantityAvailable":0.0000000000000}, > {"warehouseCode":"PO-1-CA","quantityAvailable":0.0000000000000} > ] > } > ]""" > > > # converting the json string to a Python data structure > inventory = json.loads(JSON_DATA) > > # representation in Python > #[{'inventory': [{'quantityAvailable': 0.0, 'warehouseCode': 'UT-1-US'}, > #{'quantityAvailable': 0.0, 'warehouseCode': 'KY-1-US'}, > #{'quantityAvailable': 14.0, 'warehouseCode': 'TX-1-US'}, > #{'quantityAvailable': 4.0, 'warehouseCode': 'CA-1-US'}, > #{'quantityAvailable': 1.0, 'warehouseCode': 'AB-1-CA'}, > #{'quantityAvailable': 0.0, 'warehouseCode': 'WA-1-US'}, > #{'quantityAvailable': 0.0, 'warehouseCode': 'PO-1-CA'}], > #'itemNumber': '75-5044'}] > > > # the interesting part > for items in inventory: > # get the elements in from the list > # the elements are dicts, in this case exactly one dict > print('itemNumber:', i['itemNumber']) > # nested loop iterating over the values of the key 'inventory' > for item in items['inventory']: > # the value is dict > code, qty = item['warehouseCode'],item['quantityAvailable'] > print('warehouseCode:', code, 'quantityAvailable', qty) > > > # Output > #itemNumber: 75-5044 > #warehouseCode: UT-1-US quantityAvailable 0.0 > #warehouseCode: KY-1-US quantityAvailable 0.0 > #warehouseCode: TX-1-US quantityAvailable 14.0 > #warehouseCode: CA-1-US quantityAvailable 4.0 > #warehouseCode: AB-1-CA quantityAvailable 1.0 > #warehouseCode: WA-1-US quantityAvailable 0.0 > #warehouseCode: PO-1-CA quantityAvailable 0.0 > > > Greetings > Andre Thanks Andre, that works I just needed to convert my return to a string str(DATA) From rosuav at gmail.com Thu Jun 15 08:27:40 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 22:27:40 +1000 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> Message-ID: On Thu, Jun 15, 2017 at 9:47 PM, Rhodri James wrote: >> 1) It is not secure. Check this out: >> https://stackoverflow.com/questions/1906927/xml-vulnerabilities#1907500 > XML and JSON share the vulnerabilities that come from having to parse > untrusted external input. XML then has some extra since it has extra > flexibility, like being able to specify external resources (potential attack > vectors) or entity substitution. If you don't need the extra flexibility, > feel free to use JSON, but don't for one moment think that makes you > inherently safe. Not sure what you mean about parsing untrusted external input. Suppose you build a web server that receives POST data formatted either JSON or XML. You take a puddle of bytes, and then proceed to decode them. Let's say you also decree that it can't be more than 1MB of content (if it's more than that, you reject it without parsing). Okay. What vulnerabilities are there in JSON? You could have half a million open brackets followed by half a million close brackets, but that quickly terminates Python's parser with a recursion trap. You can segfault Python if you sys.setrecursionlimit() too high, but that's your fault, not JSON's. Within the 1MB limit, this is the most memory I can imagine using: >>> data = b"[" + b"{}," * 349524 + b"{}]" That expands to 349525 empty objects, represented in Python with dictionaries, at 288 bytes apiece (using the Python 3.5 size, before the new compact representation cut that to 240 bytes). Add in the surrounding list, all 3012904 bytes of it, and the original 1MB input has expanded to 103,676,104 bytes. That's a hundred-to-one expansion - significant, but hardly the worst thing an attacker can do. In the SO link above, a demo is given where a 200KB XML payload expands to >2GB, for a more than 10K-to-one expansion. "Inherently safe"? At very least, far FAR safer. Then there are two XML attacks involving external resource access. JSON fundamentally cannot do that, ergo you are inherently safe. And the final attack involves recursion. JSON also fundamentally cannot represent any form of recursion. Winner: JSON, with 3.5 points to XML's 0.5, and that's being generous enough to give each of them half a point for the payload expansion attack. Got any other attacks against JSON? Bear in mind, you have to attack the format itself, not a buggy parser implementation (which can be corrected in a bugfix release without hassles). ChrisA From larry.martell at gmail.com Thu Jun 15 08:45:43 2017 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 15 Jun 2017 08:45:43 -0400 Subject: sqlite in 2.7 on redhat 6 Message-ID: I am trying to use sqlite $ python2.7 Python 2.7.10 (default, Feb 22 2016, 12:13:36) [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import _sqlite3 Traceback (most recent call last): File "", line 1, in ImportError: No module named _sqlite3 It's there at: /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/_sqlite3.so but that is not in my path. I tried adding /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/ to my path and then it fails with: ImportError: libpython2.7.so.1.0: cannot open shared object file: No such file or directory Isn't sqlite part of the standard lib? Shouldn't this just work? From formisc at gmail.com Thu Jun 15 08:53:26 2017 From: formisc at gmail.com (Andrew Zyman) Date: Thu, 15 Jun 2017 08:53:26 -0400 Subject: data structure In-Reply-To: References: Message-ID: Paul Thank you. In my case all "members" of a data structure are classes (except of the id). I showed the classes to highlight the requirement to access their methods as vs simple data types. I think dict of lists should work. Ideally , I hoped to access by name ( vs index), but list will do for now. Thank you. On 15 Jun 2017 03:07, "Paul Barry" wrote: > Hi Andrew. > > You start by talking about a data structure, then show code that uses > "class". Not everything in Python needs to be in a class. > > I'd look at using a simple Dictionary of lists, indexed on your ID. A > list can contain anything, so you can add your objects in there dynamically > as needed. If you need to refer to the objects by name, use a dictionary > of dictionaries. > > Examples: > > # Dict of lists. > > >>> my_objects = {} > >>> my_objects['id1'] = [] > >>> my_objects > {'id1': []} > >>> my_objects['id2'] = [] > >>> my_objects > {'id1': [], 'id2': []} > >>> my_objects['id3'] = [] > >>> my_objects > {'id1': [], 'id2': [], 'id3': []} > >>> my_objects['id2'].append('some data, but could be anything') > >>> my_objects > {'id1': [], 'id2': ['some data, but could be anything'], 'id3': []} > >>> my_objects['id2'][0] > 'some data, but could be anything' > > # Dict of dicts. > > >>> my_objects2 = {} > >>> my_objects2['id1'] = {} > >>> my_objects2['id2'] = {} > >>> my_objects2['id3'] = {} > >>> my_objects2 > {'id1': {}, 'id2': {}, 'id3': {}} > >>> my_objects2['id2']['new_key'] = 'some data, but could be anything' > >>> my_objects2 > {'id1': {}, 'id2': {'new_key': 'some data, but could be anything'}, 'id3': > {}} > >>> my_objects2['id2']['new_key'] > 'some data, but could be anything' > > I think if you concentrate on manipulating your data as opposed to trying > to write code which manipulates it, you might be better off. > > I hope this helps. > > Regards. > > Paul. > > > > > > On 15 June 2017 at 02:36, Andrew Zyman wrote: > >> Hello, >> i wonder what would be a proper data structure for something with the >> following characteristics: >> >> id - number, >> obj[a..c] - objects of various classes >> >> the idea is to be able to update certain fields of these objects initially >> getting access to the record by ID >> >> something like this ( not working ) >> >> ### code start >> >> class ClassA(object): >> a = '' >> b = '' >> def __init__(self): >> a= 'aa' >> b= 'ab' >> >> class ClassB(object): >> def __init__(self): >> self.c = 'ba' >> self.d = 'bb' >> >> def main(): >> obja = ClassA >> objb = ClassB >> >> sets = set(obja, objb) >> contracts[1] = sets >> >> print('Sets ', contracts) >> >> # with the logic like ( not working too) >> if obja.a = 'aa': >> contracts[1].obja.a = 'ABC' >> >> >> if __name__ == '__main__': >> main() >> >> >> ### code end >> >> appreciate your guidance >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > > > -- > Paul Barry, t: @barrypj - w: > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > From m.n.summerfield at googlemail.com Thu Jun 15 08:56:05 2017 From: m.n.summerfield at googlemail.com (Mark Summerfield) Date: Thu, 15 Jun 2017 05:56:05 -0700 (PDT) Subject: sqlite in 2.7 on redhat 6 In-Reply-To: References: Message-ID: On Thursday, June 15, 2017 at 1:47:00 PM UTC+1, Larry.... at gmail.com wrote: > I am trying to use sqlite > > $ python2.7 > Python 2.7.10 (default, Feb 22 2016, 12:13:36) > [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import _sqlite3 > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named _sqlite3 > > It's there at: /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/_sqlite3.so > but that is not in my path. > > I tried adding /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/ > to my path and then it fails with: > > ImportError: libpython2.7.so.1.0: cannot open shared object file: No > such file or directory > > Isn't sqlite part of the standard lib? Shouldn't this just work? Try: >>> import sqlite3 # no leading underscore From __peter__ at web.de Thu Jun 15 09:22:41 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 15 Jun 2017 15:22:41 +0200 Subject: [gettext] How to change language at run-time References: Message-ID: pozz wrote: > I know I can load multiple gettext.translation: > > it = gettext.translation('test', localedir="locale", languages=["it"]) > es = gettext.translation('test', localedir="locale", languages=["es"]) > > and install one translation at run-time when I want at a later time > (when the user selects a new language): > > it.install() > or > es.install() > > > However the problem is that strings already translated are not > translated again when a new translation is installed. So they stay at > the language selected during start-up and don't change after a new > install(). > > One solution is to restart the application, but I think there's a better > and more elegant solution. You need a way to defer the translation until the string is actually used. The documentation has a few ideas https://docs.python.org/dev/library/gettext.html#deferred-translations and here's another one -- perform the translation in the __str__ method of a custom class: import gettext class DeferredTranslation: def __init__(self, message): self.message = message def __str__(self): translate = _ if translate is DeferredTranslation: return self.message return translate(self.message) @classmethod def install(class_): import builtins builtins._ = class_ DeferredTranslation.install() message = _("Hello, world!") print(message) it = gettext.translation("test", localedir="locale", languages=["it"]) es = gettext.translation("test", localedir="locale", languages=["es"]) for language in [it, es]: language.install() print(message) From larry.martell at gmail.com Thu Jun 15 09:29:36 2017 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 15 Jun 2017 09:29:36 -0400 Subject: sqlite in 2.7 on redhat 6 In-Reply-To: References: Message-ID: On Thu, Jun 15, 2017 at 8:56 AM, Mark Summerfield via Python-list wrote: > On Thursday, June 15, 2017 at 1:47:00 PM UTC+1, Larry.... at gmail.com wrote: >> I am trying to use sqlite >> >> $ python2.7 >> Python 2.7.10 (default, Feb 22 2016, 12:13:36) >> [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> import _sqlite3 >> Traceback (most recent call last): >> File "", line 1, in >> ImportError: No module named _sqlite3 >> >> It's there at: /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/_sqlite3.so >> but that is not in my path. >> >> I tried adding /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/ >> to my path and then it fails with: >> >> ImportError: libpython2.7.so.1.0: cannot open shared object file: No >> such file or directory >> >> Isn't sqlite part of the standard lib? Shouldn't this just work? > > Try: > >>>> import sqlite3 # no leading underscore import sqlite3 Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/sqlite3/__init__.py", line 24, in from dbapi2 import * File "/usr/local/lib/python2.7/sqlite3/dbapi2.py", line 28, in from _sqlite3 import * ImportError: No module named _sqlite3 From bgailer at gmail.com Thu Jun 15 09:37:52 2017 From: bgailer at gmail.com (bob gailer) Date: Thu, 15 Jun 2017 09:37:52 -0400 Subject: Bug or intended behavior? In-Reply-To: <3c3d3d9f-9ee9-4610-a232-2d0565dcf061@googlegroups.com> References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <5d0eaeff-55b1-5989-df92-6f3c2f6a33e7@gmail.com> <3c3d3d9f-9ee9-4610-a232-2d0565dcf061@googlegroups.com> Message-ID: Sending this to docs in hopes of improving documentation of % formatting and operator precedence. Perhaps add "see 6:16 Operator precedence." On 6/3/2017 5:59 PM, Sean DiZazzo wrote: > On Friday, June 2, 2017 at 10:46:03 AM UTC-7, bob gailer wrote: >> On 6/2/2017 1:28 PM, Jussi Piitulainen wrote: >>> sean.dizazzo at gmail.com writes: >>> >>>> Can someone please explain this to me? Thanks in advance! >>>> >>>> ~Sean >>>> >>>> >>>> Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 12:39:47) >>>> [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin >>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>> print "foo %s" % 1-2 >>>> Traceback (most recent call last): >>>> File "", line 1, in >>>> TypeError: unsupported operand type(s) for -: 'str' and 'int' >>> The per cent operator has precedence over minus. Spacing is not >>> relevant. Use parentheses. >> >> In other words "foo %s" % 1 is executed, giving "1". Then "1"-2 is >> attempted giving the error. >> Also: If there is more than one conversion specifier the right argument >> to % must be a tuple. >> I usually write a tuple even if there is only one conversion specifier - >> that avoids the problem >> you encountered and makes it easy to add more values when you add more >> conversion specifiers. >> >> print "foo %s" % (1-2,) >> >> Bob Gailer > I get what it's doing, it just doesn't make much sense to me. Looking at operator precedence, I only see the % operator in regards to modulus. Nothing in regards to string formatting. Is it just a side effect of the % being overloaded in strings? Or is it intentional that it's higher precedence...and why? The documentation is unfortunately flawed. If you look at the .chm file in the docs folder under the python installation (c:\python35\docs\python351.chm as an example, and enter % in the index, you will see: % operator % formatting % interpolation note that modulo is missing! double-click operator to see the numeric operator double-click % formatting to see the string "unique built-in operation" aka interpolation. See under section 6:16 Operator precedence: "[5] The % operator is also used for string formatting; the same precedence applies." Maybe I'm making too big a deal of it. It just doesn't 'feel' right to me. Unfortunately "feel" is not a good guide to understanding programming languages. COBOL and Assembler use MOVE when it is actually COPY! Slight OT digression: The language that is best for me is APL in which there is no operator precedence to worry about. Execution is strictly right-to-left. () are used when the order of evaluation needs to be altered. I recall one person telling me that "right-to-left" was not natural. He preferred "left-to-right" as in FORTRAN. So I considered the FORTRAN statement A = B + C * D. Turns out that A * B happens first, then C + then A =. Sure looks right-to-left to me! From __peter__ at web.de Thu Jun 15 09:55:50 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 15 Jun 2017 15:55:50 +0200 Subject: sqlite in 2.7 on redhat 6 References: Message-ID: Larry Martell wrote: > On Thu, Jun 15, 2017 at 8:56 AM, Mark Summerfield via Python-list > wrote: >> On Thursday, June 15, 2017 at 1:47:00 PM UTC+1, Larry.... at gmail.com >> wrote: >>> I am trying to use sqlite >>> >>> $ python2.7 >>> Python 2.7.10 (default, Feb 22 2016, 12:13:36) >>> [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> import _sqlite3 >>> Traceback (most recent call last): >>> File "", line 1, in >>> ImportError: No module named _sqlite3 >>> >>> It's there at: >>> /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/_sqlite3.so but >>> that is not in my path. >>> >>> I tried adding /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/ >>> to my path and then it fails with: >>> >>> ImportError: libpython2.7.so.1.0: cannot open shared object file: No >>> such file or directory >>> >>> Isn't sqlite part of the standard lib? Shouldn't this just work? On linux the system sqlite3 is used. >> >> Try: >> >>>>> import sqlite3 # no leading underscore > > import sqlite3 > Traceback (most recent call last): > File "", line 1, in > File "/usr/local/lib/python2.7/sqlite3/__init__.py", line 24, in > > from dbapi2 import * > File "/usr/local/lib/python2.7/sqlite3/dbapi2.py", line 28, in > from _sqlite3 import * > ImportError: No module named _sqlite3 Is that a Python version that you compiled yourself? When the compilation finishes you get a list of missing modules. Usually the problem are missing header files. On Debian you can install the corressponding xxx-dev packages, e. g. $ sudo apt-get install libsqlite3-dev for libsqlite3. I don't know what the Redhat equivalent is. PS: There may also be a command like $ sudo apt-get build-dep python2.7 to install all build dependencies for the system Python which tend to be the same as that for a custom Python version. From rosuav at gmail.com Thu Jun 15 09:58:33 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 15 Jun 2017 23:58:33 +1000 Subject: Bug or intended behavior? In-Reply-To: References: <1aef49ce-91e5-4327-beb4-369fac8da0ac@googlegroups.com> <5d0eaeff-55b1-5989-df92-6f3c2f6a33e7@gmail.com> <3c3d3d9f-9ee9-4610-a232-2d0565dcf061@googlegroups.com> Message-ID: On Thu, Jun 15, 2017 at 11:37 PM, bob gailer wrote: > Slight OT digression: The language that is best for me is APL in which there > is no operator precedence to worry about. Execution is strictly > right-to-left. () are used when the order of evaluation needs to be altered. > I recall one person telling me that "right-to-left" was not natural. He > preferred "left-to-right" as in FORTRAN. So I considered the FORTRAN > statement A = B + C * D. Turns out that A * B happens first, then C + then A > =. Sure looks right-to-left to me! Did you learn about the basic algebraic order of operations in grade school? Multiplication happens before addition. Sane programming languages respect this. (Not counting those that don't use infix operators. Prefix or postfix operators follow different rules.) ChrisA From alister.ware at ntlworld.com Thu Jun 15 10:00:15 2017 From: alister.ware at ntlworld.com (alister) Date: Thu, 15 Jun 2017 14:00:15 GMT Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> Message-ID: On Thu, 15 Jun 2017 22:27:40 +1000, Chris Angelico wrote: > On Thu, Jun 15, 2017 at 9:47 PM, Rhodri James > wrote: >>> 1) It is not secure. Check this out: >>> https://stackoverflow.com/questions/1906927/xml- vulnerabilities#1907500 >> XML and JSON share the vulnerabilities that come from having to parse >> untrusted external input. XML then has some extra since it has extra >> flexibility, like being able to specify external resources (potential >> attack vectors) or entity substitution. If you don't need the extra >> flexibility, feel free to use JSON, but don't for one moment think that >> makes you inherently safe. > > Not sure what you mean about parsing untrusted external input. Suppose > you build a web server that receives POST data formatted either JSON or > XML. You take a puddle of bytes, and then proceed to decode them. Where it "Could" be a security issue is in Javascript. Json is designed to be legal Javascript code & therefore directly executable so no parser is posible. if a malicious site presented JavaScript code as a Json response it could expose the user. hopefully no python programmer is stupid enough to simply "exec" and data they received (whether json XML or JBCCF* ) *JBCC: Joe Blogs Custom Crap Format -- dunham: You know how real numbers are constructed from rational numbers by equivalence classes of convergent sequences? marcus: yes. From rosuav at gmail.com Thu Jun 15 10:10:58 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 16 Jun 2017 00:10:58 +1000 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> Message-ID: On Fri, Jun 16, 2017 at 12:00 AM, alister wrote: > On Thu, 15 Jun 2017 22:27:40 +1000, Chris Angelico wrote: > >> On Thu, Jun 15, 2017 at 9:47 PM, Rhodri James >> wrote: >>>> 1) It is not secure. Check this out: >>>> https://stackoverflow.com/questions/1906927/xml- > vulnerabilities#1907500 >>> XML and JSON share the vulnerabilities that come from having to parse >>> untrusted external input. XML then has some extra since it has extra >>> flexibility, like being able to specify external resources (potential >>> attack vectors) or entity substitution. If you don't need the extra >>> flexibility, feel free to use JSON, but don't for one moment think that >>> makes you inherently safe. >> >> Not sure what you mean about parsing untrusted external input. Suppose >> you build a web server that receives POST data formatted either JSON or >> XML. You take a puddle of bytes, and then proceed to decode them. > > Where it "Could" be a security issue is in Javascript. > > Json is designed to be legal Javascript code & therefore directly > executable so no parser is posible. > "no parser is possible"??? https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse If you're stupid enough to eval JSON instead of using JSON.parse(), you deserve all you get. That's not a fault with JSON. ChrisA From larry.martell at gmail.com Thu Jun 15 10:14:14 2017 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 15 Jun 2017 10:14:14 -0400 Subject: sqlite in 2.7 on redhat 6 In-Reply-To: References: Message-ID: On Thu, Jun 15, 2017 at 9:55 AM, Peter Otten <__peter__ at web.de> wrote: > Larry Martell wrote: > >> On Thu, Jun 15, 2017 at 8:56 AM, Mark Summerfield via Python-list >> wrote: >>> On Thursday, June 15, 2017 at 1:47:00 PM UTC+1, Larry.... at gmail.com >>> wrote: >>>> I am trying to use sqlite >>>> >>>> $ python2.7 >>>> Python 2.7.10 (default, Feb 22 2016, 12:13:36) >>>> [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>> >>> import _sqlite3 >>>> Traceback (most recent call last): >>>> File "", line 1, in >>>> ImportError: No module named _sqlite3 >>>> >>>> It's there at: >>>> /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/_sqlite3.so but >>>> that is not in my path. >>>> >>>> I tried adding /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/ >>>> to my path and then it fails with: >>>> >>>> ImportError: libpython2.7.so.1.0: cannot open shared object file: No >>>> such file or directory >>>> >>>> Isn't sqlite part of the standard lib? Shouldn't this just work? > > On linux the system sqlite3 is used. I tried building and installing sqlite from source and that did not solve the problem. > >>> >>> Try: >>> >>>>>> import sqlite3 # no leading underscore >> >> import sqlite3 >> Traceback (most recent call last): >> File "", line 1, in >> File "/usr/local/lib/python2.7/sqlite3/__init__.py", line 24, in >> >> from dbapi2 import * >> File "/usr/local/lib/python2.7/sqlite3/dbapi2.py", line 28, in >> from _sqlite3 import * >> ImportError: No module named _sqlite3 > > Is that a Python version that you compiled yourself? No I installed it with pip install python2.7 > When the compilation > finishes you get a list of missing modules. Usually the problem are missing > header files. On Debian you can install the corressponding xxx-dev packages, > e. g. > > $ sudo apt-get install libsqlite3-dev > > for libsqlite3. I don't know what the Redhat equivalent is. > > PS: There may also be a command like > > $ sudo apt-get build-dep python2.7 > > to install all build dependencies for the system Python which tend to be the > same as that for a custom Python version. From __peter__ at web.de Thu Jun 15 10:35:49 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 15 Jun 2017 16:35:49 +0200 Subject: sqlite in 2.7 on redhat 6 References: Message-ID: Larry Martell wrote: >> On linux the system sqlite3 is used. > > I tried building and installing sqlite from source and that did not > solve the problem. You misunderstood: the problem is not sqlite3 it's that python needs sqlite3's header files. >> Is that a Python version that you compiled yourself? > > No I installed it with pip install python2.7 I'd never have thought of trying that -- but still: did that download binary blobs or did it invoke the C compiler? If the latter, and pip isn't smart enough to get the sqlite3 headers you need to install them by hand before letting pip do its magic. From larry.martell at gmail.com Thu Jun 15 10:49:48 2017 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 15 Jun 2017 10:49:48 -0400 Subject: sqlite in 2.7 on redhat 6 In-Reply-To: References: Message-ID: On Thu, Jun 15, 2017 at 10:35 AM, Peter Otten <__peter__ at web.de> wrote: > Larry Martell wrote: > >>> On linux the system sqlite3 is used. >> >> I tried building and installing sqlite from source and that did not >> solve the problem. > > You misunderstood: the problem is not sqlite3 it's that python needs > sqlite3's header files. > >>> Is that a Python version that you compiled yourself? >> >> No I installed it with pip install python2.7 > > I'd never have thought of trying that -- but still: did that download binary > blobs or did it invoke the C compiler? Sorry I mistyped - it wasn't pip it was yum. > If the latter, and pip isn't smart > enough to get the sqlite3 headers you need to install them by hand before > letting pip do its magic. From ned at nedbatchelder.com Thu Jun 15 10:55:59 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 15 Jun 2017 07:55:59 -0700 (PDT) Subject: Does Python need Javascript? In-Reply-To: References: <594233cc$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thursday, June 15, 2017 at 5:55:07 AM UTC-4, Gregory Ewing wrote: > Chris Angelico wrote: > > There've been a number of attempts, over the years, to make a > > sandboxed Python interpreter, where you can run untrusted code. But if > > Python came with a JS interpreter, it would be possible to run > > untrusted JS code, with Python functioning as a gatekeeper. > > If that would provide sufficient sandboxing, then how about > writing a Python interpreter in Python to get a sandboxed > Python? PyPy has a sandboxed mode, where every system call is routed through a controller process than can decide what to do with it. --Ned. From __peter__ at web.de Thu Jun 15 11:04:49 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 15 Jun 2017 17:04:49 +0200 Subject: sqlite in 2.7 on redhat 6 References: Message-ID: Larry Martell wrote: > Sorry I mistyped - it wasn't pip it was yum. OK, I'm out then. Looks like what works for Debian derivatives is not easily transferable to Redhead... From gheskett at shentel.net Thu Jun 15 11:25:56 2017 From: gheskett at shentel.net (Gene Heskett) Date: Thu, 15 Jun 2017 11:25:56 -0400 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: <87fuf1def1.fsf@elektro.pacujo.net> References: <87fuf1def1.fsf@elektro.pacujo.net> Message-ID: <201706151125.56708.gheskett@shentel.net> On Thursday 15 June 2017 01:10:26 Marko Rauhamaa wrote: > Chris Angelico : > > XML is thus poorly suited to *most* forms of data, > > Correct. > > > > > > > aa > > qq > > qw > > qe > > as > > > > > > What does this represent? A generic XML parser has to cope with it. > > I gave this to a few XML-to-JSON converters, and they all > > interpreted it as some variant of > > {"asdf":["aa","as"],"qwer":["qq","qw","qe"]} > > It's worse than that. For a *generic* parser, it should be something > like: > > { > "spam" : [ > " \n", > { "asdf" : [ "aa" ] }, > " \n", > { "qwer" : [ "qq" ] }, > " \n", > { "qwer" : [ "qw" ] }, > " \n", > { "qwer" : [ "qe" ] }, > " \n", > { "asdf" : [ "as" ] }, > "\n" > ] > } > > > XML just needs to die. > > Thankfully, that seems to be happening. Slowly please, it is still being heavily used in the custom gui creation field. We would love to have gladevcp back in a usable state, but it was dropped by gtk3 and has never been given any TLC since, so those of us creating industrial gui's are stuck doing it in pyvcp and xml. If you intend to kill off xml, then we had better have another, even prettier replacement already available. We don't today, have anything that can replace pyvcp translating a .xml in its broad utility. > Marko Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From pozzugno at gmail.com Thu Jun 15 11:58:34 2017 From: pozzugno at gmail.com (pozz) Date: Thu, 15 Jun 2017 17:58:34 +0200 Subject: [gettext] How to change language at run-time In-Reply-To: References: Message-ID: Il 15/06/2017 15:22, Peter Otten ha scritto: > pozz wrote: > >> I know I can load multiple gettext.translation: >> >> it = gettext.translation('test', localedir="locale", languages=["it"]) >> es = gettext.translation('test', localedir="locale", languages=["es"]) >> >> and install one translation at run-time when I want at a later time >> (when the user selects a new language): >> >> it.install() >> or >> es.install() >> >> >> However the problem is that strings already translated are not >> translated again when a new translation is installed. So they stay at >> the language selected during start-up and don't change after a new >> install(). >> >> One solution is to restart the application, but I think there's a better >> and more elegant solution. > > You need a way to defer the translation until the string is actually used. > The documentation has a few ideas > > https://docs.python.org/dev/library/gettext.html#deferred-translations > > and here's another one -- perform the translation in the __str__ method of > a custom class: > [...] It's a nice trick. However you will have a string that isn't a string, but a class. I think you can't use the class everywhere you can use a string. For example, len() can't be called. From kushal at locationd.net Thu Jun 15 12:20:43 2017 From: kushal at locationd.net (Kushal Kumaran) Date: Thu, 15 Jun 2017 09:20:43 -0700 Subject: sqlite in 2.7 on redhat 6 In-Reply-To: (Larry Martell's message of "Thu, 15 Jun 2017 10:49:48 -0400") References: Message-ID: <87r2ylkysk.fsf@vanadium.community.veritas.com> Larry Martell writes: > On Thu, Jun 15, 2017 at 10:35 AM, Peter Otten <__peter__ at web.de> wrote: >> Larry Martell wrote: >> >>>> On linux the system sqlite3 is used. >>> >>> I tried building and installing sqlite from source and that did not >>> solve the problem. >> >> You misunderstood: the problem is not sqlite3 it's that python needs >> sqlite3's header files. >> >>>> Is that a Python version that you compiled yourself? >>> >>> No I installed it with pip install python2.7 >> >> I'd never have thought of trying that -- but still: did that download binary >> blobs or did it invoke the C compiler? > > Sorry I mistyped - it wasn't pip it was yum. > Are you sure that is the python you are running? The python provided by the OS repositories would have installed python at /usr/bin/python (possibly with additional names such as /usr/bin/python2.7). The exception you posted earlier is looking at files in /usr/local and /opt/rh, neither of which would be searched by the system-installed python by default. It is possible (likely?) that /usr/local/bin is earlier in your $PATH than /usr/bin, which is resulting in some /usr/local/bin/python shadowing the system python. If so, run the system python with an absolute path like /usr/bin/python and see if you are able to import sqlite3. >> If the latter, and pip isn't smart >> enough to get the sqlite3 headers you need to install them by hand before >> letting pip do its magic. -- regards, kushal From __peter__ at web.de Thu Jun 15 12:40:38 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 15 Jun 2017 18:40:38 +0200 Subject: [gettext] How to change language at run-time References: Message-ID: pozz wrote: > Il 15/06/2017 15:22, Peter Otten ha scritto: >> pozz wrote: >> >>> I know I can load multiple gettext.translation: >>> >>> it = gettext.translation('test', localedir="locale", >>> languages=["it"]) es = gettext.translation('test', >>> localedir="locale", languages=["es"]) >>> >>> and install one translation at run-time when I want at a later time >>> (when the user selects a new language): >>> >>> it.install() >>> or >>> es.install() >>> >>> >>> However the problem is that strings already translated are not >>> translated again when a new translation is installed. So they stay at >>> the language selected during start-up and don't change after a new >>> install(). >>> >>> One solution is to restart the application, but I think there's a better >>> and more elegant solution. >> >> You need a way to defer the translation until the string is actually >> used. The documentation has a few ideas >> >> https://docs.python.org/dev/library/gettext.html#deferred-translations >> >> and here's another one -- perform the translation in the __str__ method >> of a custom class: > > [...] > > > It's a nice trick. However you will have a string that isn't a string, > but a class. I think you can't use the class everywhere you can use a > string. For example, len() can't be called. len() could be implemented as class DeferredTranslation: ... def __len__(self): return len(str(self)) and usually I would expect that you only need a small subset of the str methods for localized text. However, when you switch languages between the len() and str() calls you will certainly make a mess... From larry.martell at gmail.com Thu Jun 15 13:22:42 2017 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 15 Jun 2017 13:22:42 -0400 Subject: sqlite in 2.7 on redhat 6 In-Reply-To: <87r2ylkysk.fsf@vanadium.community.veritas.com> References: <87r2ylkysk.fsf@vanadium.community.veritas.com> Message-ID: On Thu, Jun 15, 2017 at 12:20 PM, Kushal Kumaran wrote: > Larry Martell writes: > >> On Thu, Jun 15, 2017 at 10:35 AM, Peter Otten <__peter__ at web.de> wrote: >>> Larry Martell wrote: >>> >>>>> On linux the system sqlite3 is used. >>>> >>>> I tried building and installing sqlite from source and that did not >>>> solve the problem. >>> >>> You misunderstood: the problem is not sqlite3 it's that python needs >>> sqlite3's header files. >>> >>>>> Is that a Python version that you compiled yourself? >>>> >>>> No I installed it with pip install python2.7 >>> >>> I'd never have thought of trying that -- but still: did that download binary >>> blobs or did it invoke the C compiler? >> >> Sorry I mistyped - it wasn't pip it was yum. >> > > Are you sure that is the python you are running? The python provided by > the OS repositories would have installed python at /usr/bin/python > (possibly with additional names such as /usr/bin/python2.7). The > exception you posted earlier is looking at files in /usr/local and > /opt/rh, neither of which would be searched by the system-installed > python by default. > > It is possible (likely?) that /usr/local/bin is earlier in your $PATH > than /usr/bin, which is resulting in some /usr/local/bin/python > shadowing the system python. If so, run the system python with an > absolute path like /usr/bin/python and see if you are able to import > sqlite3. /usr/bin/python2.7 is a symlink to /usr/local/bin/python2.7: $ ls -l /usr/bin/python2.7 lrwxrwxrwx. 1 root root 24 Apr 22 2016 /usr/bin/python2.7 -> /usr/local/bin/python2.7 On Redhat 6 the system python is 2.6. You cannot change that as it breaks yum. /usr/local/bin/ is earlier in my path then /usr/bin. But I used python2.7 all the time here and other packages work fine. What led me down this rabbit hole was that I am trying to use pypolibox and that is failing with this traceback: Traceback (most recent call last): File "/usr/local/bin/pypolibox", line 9, in load_entry_point('pypolibox==1.0.2', 'console_scripts', 'pypolibox')() File "/usr/local/lib/python2.7/site-packages/distribute-0.6.35-py2.7.egg/pkg_resources.py", line 343, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/usr/local/lib/python2.7/site-packages/distribute-0.6.35-py2.7.egg/pkg_resources.py", line 2309, in load_entry_point return ep.load() File "/usr/local/lib/python2.7/site-packages/distribute-0.6.35-py2.7.egg/pkg_resources.py", line 2015, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/usr/local/lib/python2.7/site-packages/pypolibox/pypolibox.py", line 13, in from nltk.featstruct import Feature File "/usr/local/lib/python2.7/site-packages/nltk/__init__.py", line 137, in from nltk.stem import * File "/usr/local/lib/python2.7/site-packages/nltk/stem/__init__.py", line 29, in from nltk.stem.snowball import SnowballStemmer File "/usr/local/lib/python2.7/site-packages/nltk/stem/snowball.py", line 26, in from nltk.corpus import stopwords File "/usr/local/lib/python2.7/site-packages/nltk/corpus/__init__.py", line 66, in from nltk.corpus.reader import * File "/usr/local/lib/python2.7/site-packages/nltk/corpus/reader/__init__.py", line 105, in from nltk.corpus.reader.panlex_lite import * File "/usr/local/lib/python2.7/site-packages/nltk/corpus/reader/panlex_lite.py", line 15, in import sqlite3 File "/usr/local/lib/python2.7/sqlite3/__init__.py", line 24, in from dbapi2 import * File "/usr/local/lib/python2.7/sqlite3/dbapi2.py", line 28, in from _sqlite3 import * ImportError: No module named _sqlite3 From kushal at locationd.net Thu Jun 15 13:44:59 2017 From: kushal at locationd.net (Kushal Kumaran) Date: Thu, 15 Jun 2017 10:44:59 -0700 Subject: sqlite in 2.7 on redhat 6 In-Reply-To: (Larry Martell's message of "Thu, 15 Jun 2017 13:22:42 -0400") References: <87r2ylkysk.fsf@vanadium.community.veritas.com> Message-ID: <87mv99kuw4.fsf@vanadium.community.veritas.com> Larry Martell writes: > On Thu, Jun 15, 2017 at 12:20 PM, Kushal Kumaran wrote: >> Larry Martell writes: >> >>> On Thu, Jun 15, 2017 at 10:35 AM, Peter Otten <__peter__ at web.de> wrote: >>>> Larry Martell wrote: >>>> >>>>>> On linux the system sqlite3 is used. >>>>> >>>>> I tried building and installing sqlite from source and that did not >>>>> solve the problem. >>>> >>>> You misunderstood: the problem is not sqlite3 it's that python needs >>>> sqlite3's header files. >>>> >>>>>> Is that a Python version that you compiled yourself? >>>>> >>>>> No I installed it with pip install python2.7 >>>> >>>> I'd never have thought of trying that -- but still: did that download binary >>>> blobs or did it invoke the C compiler? >>> >>> Sorry I mistyped - it wasn't pip it was yum. >>> >> >> Are you sure that is the python you are running? The python provided by >> the OS repositories would have installed python at /usr/bin/python >> (possibly with additional names such as /usr/bin/python2.7). The >> exception you posted earlier is looking at files in /usr/local and >> /opt/rh, neither of which would be searched by the system-installed >> python by default. >> >> It is possible (likely?) that /usr/local/bin is earlier in your $PATH >> than /usr/bin, which is resulting in some /usr/local/bin/python >> shadowing the system python. If so, run the system python with an >> absolute path like /usr/bin/python and see if you are able to import >> sqlite3. > > /usr/bin/python2.7 is a symlink to /usr/local/bin/python2.7: > > $ ls -l /usr/bin/python2.7 > > lrwxrwxrwx. 1 root root 24 Apr 22 2016 /usr/bin/python2.7 -> > /usr/local/bin/python2.7 > > On Redhat 6 the system python is 2.6. You cannot change that as it breaks yum. > Then the question is again, how did you install python2.7. Which yum repository provided you this kind of python2.7 package? You should contact them to check how they've built this python, and whether they can include sqlite3 in their build. > /usr/local/bin/ is earlier in my path then /usr/bin. But I used > python2.7 all the time here and other packages work fine. > > What led me down this rabbit hole was that I am trying to use > pypolibox and that is failing with this traceback: > > Traceback (most recent call last): > File "/usr/local/bin/pypolibox", line 9, in > load_entry_point('pypolibox==1.0.2', 'console_scripts', 'pypolibox')() > File "/usr/local/lib/python2.7/site-packages/distribute-0.6.35-py2.7.egg/pkg_resources.py", > line 343, in load_entry_point > return get_distribution(dist).load_entry_point(group, name) > File "/usr/local/lib/python2.7/site-packages/distribute-0.6.35-py2.7.egg/pkg_resources.py", > line 2309, in load_entry_point > return ep.load() > File "/usr/local/lib/python2.7/site-packages/distribute-0.6.35-py2.7.egg/pkg_resources.py", > line 2015, in load > entry = __import__(self.module_name, globals(),globals(), ['__name__']) > File "/usr/local/lib/python2.7/site-packages/pypolibox/pypolibox.py", > line 13, in > from nltk.featstruct import Feature > File "/usr/local/lib/python2.7/site-packages/nltk/__init__.py", line > 137, in > from nltk.stem import * > File "/usr/local/lib/python2.7/site-packages/nltk/stem/__init__.py", > line 29, in > from nltk.stem.snowball import SnowballStemmer > File "/usr/local/lib/python2.7/site-packages/nltk/stem/snowball.py", > line 26, in > from nltk.corpus import stopwords > File "/usr/local/lib/python2.7/site-packages/nltk/corpus/__init__.py", > line 66, in > from nltk.corpus.reader import * > File "/usr/local/lib/python2.7/site-packages/nltk/corpus/reader/__init__.py", > line 105, in > from nltk.corpus.reader.panlex_lite import * > File "/usr/local/lib/python2.7/site-packages/nltk/corpus/reader/panlex_lite.py", > line 15, in > import sqlite3 > File "/usr/local/lib/python2.7/sqlite3/__init__.py", line 24, in > from dbapi2 import * > File "/usr/local/lib/python2.7/sqlite3/dbapi2.py", line 28, in > from _sqlite3 import * > ImportError: No module named _sqlite3 Your python was built without sqlite3 support. If your yum repo provider is unable to rebuild with sqlite3, you can do so yourself. Make sure the sqlite-devel package is installed so your python build will use it. -- regards, kushal From python at lucidity.plus.com Thu Jun 15 16:17:05 2017 From: python at lucidity.plus.com (Erik) Date: Thu, 15 Jun 2017 21:17:05 +0100 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> Message-ID: <194b245e-f4dc-cc38-5a2d-b772574d2095@lucidity.plus.com> On 15/06/17 15:10, Chris Angelico wrote: > On Fri, Jun 16, 2017 at 12:00 AM, alister wrote: >> Json is designed to be legal Javascript code & therefore directly >> executable so no parser is posible. >> > > "no parser is possible"??? I *think* alister meant "so it is possible to not use a parser [library]" (i.e., parse the stream using JavaScript's parser via eval() - though I agree with everyone else who has said this should never be done). I may be wrong about what alister meant, but the language reminds me of a German colleague of mine from a few years back who wrote in some API update documentation "Specifying parameter X is no longer an option". What he meant was "is now mandatory" or "is no longer optional". To a native English speaker, it reads as "can no longer be used" which is the opposite of what he meant ... E. From grant.b.edwards at gmail.com Thu Jun 15 16:58:16 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 15 Jun 2017 20:58:16 +0000 (UTC) Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> <194b245e-f4dc-cc38-5a2d-b772574d2095@lucidity.plus.com> Message-ID: On 2017-06-15, Erik wrote: > On 15/06/17 15:10, Chris Angelico wrote: >> On Fri, Jun 16, 2017 at 12:00 AM, alister wrote: >>> Json is designed to be legal Javascript code & therefore directly >>> executable so no parser is posible. >>> >> >> "no parser is possible"??? > > I *think* alister meant "so it is possible to not use a parser > [library]" (i.e., parse the stream using JavaScript's parser via eval() > - though I agree with everyone else who has said this should never be done). The old operator order/precedence issue strikes again... (no parser) is possible vs. no (parser is possible) -- Grant Edwards grant.b.edwards Yow! I'm definitely not at in Omaha! gmail.com From larry.martell at gmail.com Thu Jun 15 17:07:47 2017 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 15 Jun 2017 17:07:47 -0400 Subject: sqlite in 2.7 on redhat 6 In-Reply-To: <87mv99kuw4.fsf@vanadium.community.veritas.com> References: <87r2ylkysk.fsf@vanadium.community.veritas.com> <87mv99kuw4.fsf@vanadium.community.veritas.com> Message-ID: On Thu, Jun 15, 2017 at 1:44 PM, Kushal Kumaran wrote: > Your python was built without sqlite3 support. If your yum repo > provider is unable to rebuild with sqlite3, you can do so yourself. > Make sure the sqlite-devel package is installed so your python build > will use it. Decided to do this in a docker container running an OS where 2.7 is the default and has sqlite. Thanks everyone for their advice and help. From rosuav at gmail.com Thu Jun 15 17:38:39 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 16 Jun 2017 07:38:39 +1000 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: <194b245e-f4dc-cc38-5a2d-b772574d2095@lucidity.plus.com> References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> <194b245e-f4dc-cc38-5a2d-b772574d2095@lucidity.plus.com> Message-ID: On Fri, Jun 16, 2017 at 6:17 AM, Erik wrote: > On 15/06/17 15:10, Chris Angelico wrote: >> >> On Fri, Jun 16, 2017 at 12:00 AM, alister >> wrote: >>> >>> Json is designed to be legal Javascript code & therefore directly >>> executable so no parser is posible. >>> >> >> "no parser is possible"??? > > > I *think* alister meant "so it is possible to not use a parser [library]" > (i.e., parse the stream using JavaScript's parser via eval() - though I > agree with everyone else who has said this should never be done). > > I may be wrong about what alister meant, but the language reminds me of a > German colleague of mine from a few years back who wrote in some API update > documentation "Specifying parameter X is no longer an option". What he meant > was "is now mandatory" or "is no longer optional". To a native English > speaker, it reads as "can no longer be used" which is the opposite of what > he meant ... Ohhh, got it. Well, that's technically true, but it's still not a fault of JSON. A proper parser exists in most web languages' standard libraries, and can probably be picked up for any language you like. So I still stand by my declaration ChrisA From rosuav at gmail.com Thu Jun 15 17:43:28 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 16 Jun 2017 07:43:28 +1000 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> <194b245e-f4dc-cc38-5a2d-b772574d2095@lucidity.plus.com> Message-ID: On Fri, Jun 16, 2017 at 6:58 AM, Grant Edwards wrote: > On 2017-06-15, Erik wrote: >> On 15/06/17 15:10, Chris Angelico wrote: >>> On Fri, Jun 16, 2017 at 12:00 AM, alister wrote: >>>> Json is designed to be legal Javascript code & therefore directly >>>> executable so no parser is posible. >>>> >>> >>> "no parser is possible"??? >> >> I *think* alister meant "so it is possible to not use a parser >> [library]" (i.e., parse the stream using JavaScript's parser via eval() >> - though I agree with everyone else who has said this should never be done). > > The old operator order/precedence issue strikes again... > > (no parser) is possible > > vs. > > no (parser is possible) Rewording: JSON is a subset of JavaScript syntax, and therefore can potentially be evaluated without a parser. So, yeah, sure. You can create an HTML page like this: """ """ % json.dumps(some_object) Sure, nice. But if you then say "JSON is vulnerable because you can evaluate it as JS", you're completely missing the point. Any time you get untrusted data, you're going to get it as either a string or an array of bytes, and either way, it's just as easy to JSON.parse() it as to eval() it. ChrisA From jladasky at itu.edu Thu Jun 15 18:24:40 2017 From: jladasky at itu.edu (jladasky at itu.edu) Date: Thu, 15 Jun 2017 15:24:40 -0700 (PDT) Subject: Developers Who Use Spaces Make More Money! Message-ID: This is hilarious, I have to share: https://stackoverflow.blog/2017/06/15/developers-use-spaces-make-money-use-tabs/ Thanks to Guido for making us all richer! From christopher_reimer at icloud.com Thu Jun 15 18:35:25 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Thu, 15 Jun 2017 15:35:25 -0700 Subject: Developers Who Use Spaces Make More Money! In-Reply-To: References: Message-ID: > On Jun 15, 2017, at 3:24 PM, jladasky at itu.edu wrote: > > This is hilarious, I have to share: > > https://stackoverflow.blog/2017/06/15/developers-use-spaces-make-money-use-tabs/ > > Thanks to Guido for making us all richer! One commentator on a tech website admonished programmers for wasting time by pressing the space bar four times instead of using tab. O_o From rosuav at gmail.com Thu Jun 15 18:42:05 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 16 Jun 2017 08:42:05 +1000 Subject: Developers Who Use Spaces Make More Money! In-Reply-To: References: Message-ID: On Fri, Jun 16, 2017 at 8:24 AM, wrote: > This is hilarious, I have to share: > > https://stackoverflow.blog/2017/06/15/developers-use-spaces-make-money-use-tabs/ > > Thanks to Guido for making us all richer! Bah, that's just noise. There may appear to be a small difference between those who use tabs and those who use spaces, but look at the numbers: whichever one you use, you're going to make FAR more money than people who don't use any whitespace in their code. Search your feelings, you know this to be true... ChrisA From steve+python at pearwood.info Thu Jun 15 20:43:19 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 16 Jun 2017 10:43:19 +1000 Subject: Developers Who Use Spaces Make More Money! References: Message-ID: <594329a9$0$1596$c3e8da3$5496439d@news.astraweb.com> On Fri, 16 Jun 2017 08:42 am, Chris Angelico wrote: > On Fri, Jun 16, 2017 at 8:24 AM, wrote: >> This is hilarious, I have to share: >> >> https://stackoverflow.blog/2017/06/15/developers-use-spaces-make-money-use-tabs/ >> >> Thanks to Guido for making us all richer! > > Bah, that's just noise. There may appear to be a small difference > between those who use tabs and those who use spaces, but look at the > numbers: whichever one you use, you're going to make FAR more money > than people who don't use any whitespace in their code. Oh man... can you imagine how much money there is to be made by eliminating all non-whitespace from your code? http://progopedia.com/language/whitespace/ -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From larry.martell at gmail.com Thu Jun 15 22:51:14 2017 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 15 Jun 2017 22:51:14 -0400 Subject: Developers Who Use Spaces Make More Money! In-Reply-To: References: Message-ID: On Thu, Jun 15, 2017 at 6:35 PM, Christopher Reimer wrote: >> On Jun 15, 2017, at 3:24 PM, jladasky at itu.edu wrote: >> >> This is hilarious, I have to share: >> >> https://stackoverflow.blog/2017/06/15/developers-use-spaces-make-money-use-tabs/ >> >> Thanks to Guido for making us all richer! > > One commentator on a tech website admonished programmers for wasting time by pressing the space bar four times instead of using tab. O_o Not in vi with set autoindent set expandtab From ian.g.kelly at gmail.com Thu Jun 15 23:11:41 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 15 Jun 2017 21:11:41 -0600 Subject: Developers Who Use Spaces Make More Money! In-Reply-To: References: Message-ID: On Thu, Jun 15, 2017 at 8:51 PM, Larry Martell wrote: > On Thu, Jun 15, 2017 at 6:35 PM, Christopher Reimer > wrote: >> One commentator on a tech website admonished programmers for wasting time by pressing the space bar four times instead of using tab. O_o > > Not in vi with > > set autoindent > set expandtab Or in any halfway-decent editor with reasonable indentation settings. From ben+python at benfinney.id.au Fri Jun 16 01:53:31 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 16 Jun 2017 15:53:31 +1000 Subject: Customise the virtualenv `activate` script References: <85mv9cwsy0.fsf@benfinney.id.au> <20170613075057.GA51505@cskk.homeip.net> Message-ID: <85h8zgv5pg.fsf@benfinney.id.au> Cameron Simpson writes: > I must admit my initial preference would be the differently named > wrapper. Surely users of the codebase will be invoking stuff via > something opaque which sources the requisite things? That ?something opaque? is the ?$VENV/bin/activate? script; many people who join the team will already know that, and I'm trying to make use of that existing convention. > Actually, on trying to write something simple and flexible, since once > made the venv is basicly state WRT the activate script, I'm leaning > towards hacking the activate script, probably by keeping a distinct > file off the the side and modifying activate to source it. Yeah, I'd much prefer to be told there's a hook to use, so that someone who creates a standard Python virtualenv the conventional way will not need to then hack that virtualenv. -- \ ?I distrust those people who know so well what God wants them | `\ to do to their fellows, because it always coincides with their | _o__) own desires.? ?Susan Brownell Anthony, 1896 | Ben Finney From ben+python at benfinney.id.au Fri Jun 16 01:56:47 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 16 Jun 2017 15:56:47 +1000 Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> Message-ID: <85d1a4v5k0.fsf@benfinney.id.au> alister writes: > Json is designed to be legal Javascript code & therefore directly > executable so no parser is posible. JSON is designed to be *a strictly limited subset* of legal JavaScript that only defines data structures. The explicit goal is that it is statically parseable as non-executable data. -- \ ?The power of accurate observation is frequently called | `\ cynicism by those who don't have it.? ?George Bernard Shaw | _o__) | Ben Finney From dieter at handshake.de Fri Jun 16 02:23:08 2017 From: dieter at handshake.de (dieter) Date: Fri, 16 Jun 2017 08:23:08 +0200 Subject: [gettext] How to change language at run-time References: Message-ID: <87lgosxxgz.fsf@handshake.de> pozz writes: > I know I can load multiple gettext.translation: > > it = gettext.translation('test', localedir="locale", languages=["it"]) > es = gettext.translation('test', localedir="locale", languages=["es"]) > > and install one translation at run-time when I want at a later time > (when the user selects a new language): > > it.install() > or > es.install() > > > However the problem is that strings already translated are not > translated again when a new translation is installed. So they stay at > the language selected during start-up and don't change after a new > install(). I know the internationalization/localisation of the web application framework "Zope". There, it is completely natural, that the individual request determines the target language. This is handled as follows: things to be localized are represented by so called "message_id"s. They behave somehow as unicode objects, but, of course, they are more complex; especially, they may encapsulate parameters to be incorparated in the translation. When a "message_id" is localized, the target language is taken from the context (i.e. the current request). It returns a unicode string but does not change the "message_id". From jaidip.81 at gmail.com Fri Jun 16 05:06:25 2017 From: jaidip.81 at gmail.com (JAIDIP patel) Date: Fri, 16 Jun 2017 02:06:25 -0700 (PDT) Subject: little help in homework required Message-ID: I just wrote the code with Python 3.6.1, I am just a beginner in python. while compiling having below error, can anyone help me in this TypeError: a bytes-like object is required, not 'str' The code I wrote: import socket mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) mysock.connect(('data.pr4e.org', 80)) mysock.send('GET http://data.pr4e.org/intro-short.txt HTTP/1.0\n\n') while True: data =x.recv(512) if ( len(data) < 1 ) : break print (data) mysock.close() From lutz.horn at posteo.de Fri Jun 16 05:39:55 2017 From: lutz.horn at posteo.de (Lutz Horn) Date: Fri, 16 Jun 2017 11:39:55 +0200 Subject: little help in homework required In-Reply-To: References: Message-ID: <07d5274e4ef73124e490727bf3920057@posteo.de> Hi, > TypeError: a bytes-like object is required, not 'str' ... > mysock.send('GET http://data.pr4e.org/intro-short.txt HTTP/1.0\n\n') Here you must encode the str as bytes: mysock.send('GET http://data.pr4e.org/intro-short.txt HTTP/1.0\n\n'.encode("UTF-8")) The actual encoding does not matter in your case since you only have ASCII. > data =x.recv(512) This gives the next error: NameError: name 'x' is not defined What is x supposed to be? Regards Lutz From rosuav at gmail.com Fri Jun 16 06:21:38 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 16 Jun 2017 20:21:38 +1000 Subject: little help in homework required In-Reply-To: References: Message-ID: On Fri, Jun 16, 2017 at 7:06 PM, JAIDIP patel wrote: > The code I wrote: > > import socket > > mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > mysock.connect(('data.pr4e.org', 80)) > mysock.send('GET http://data.pr4e.org/intro-short.txt HTTP/1.0\n\n') > while True: > data =x.recv(512) > if ( len(data) < 1 ) : > break > print (data) > > mysock.close() Lutz has already suggested one solution, but here's an alternative: you can use a byte-string literal. mysock.send(b'GET http://data.pr4e.org/intro-short.txt HTTP/1.0\n\n') While you're at it, though, I recommend complying with protocol correctly: mysock.send(b'GET http://data.pr4e.org/intro-short.txt HTTP/1.0\r\n\r\n') The HTTP specification says that lines should end with CR LF. You can actually simplify your condition, too: if not data: This is considered more "Pythonic" - that is to say, it's more the way that experienced Python programmers work. Consider it points for style. :) Hope that helps! ChrisA From alister.ware at ntlworld.com Fri Jun 16 06:58:49 2017 From: alister.ware at ntlworld.com (alister) Date: Fri, 16 Jun 2017 10:58:49 GMT Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> Message-ID: On Fri, 16 Jun 2017 00:10:58 +1000, Chris Angelico wrote: > On Fri, Jun 16, 2017 at 12:00 AM, alister > wrote: >> On Thu, 15 Jun 2017 22:27:40 +1000, Chris Angelico wrote: >> >>> On Thu, Jun 15, 2017 at 9:47 PM, Rhodri James >>> wrote: >>>>> 1) It is not secure. Check this out: >>>>> https://stackoverflow.com/questions/1906927/xml- >> vulnerabilities#1907500 >>>> XML and JSON share the vulnerabilities that come from having to parse >>>> untrusted external input. XML then has some extra since it has extra >>>> flexibility, like being able to specify external resources (potential >>>> attack vectors) or entity substitution. If you don't need the extra >>>> flexibility, feel free to use JSON, but don't for one moment think >>>> that makes you inherently safe. >>> >>> Not sure what you mean about parsing untrusted external input. Suppose >>> you build a web server that receives POST data formatted either JSON >>> or XML. You take a puddle of bytes, and then proceed to decode them. >> >> Where it "Could" be a security issue is in Javascript. >> >> Json is designed to be legal Javascript code & therefore directly >> executable so no parser is posible. >> >> > "no parser is possible"??? > > https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/ Global_Objects/JSON/parse > > If you're stupid enough to eval JSON instead of using JSON.parse(), > you deserve all you get. That's not a fault with JSON. > > ChrisA i meant possible to use without a parser , sorry -- Dijkstra probably hates me (Linus Torvalds, in kernel/sched.c) From alister.ware at ntlworld.com Fri Jun 16 07:03:47 2017 From: alister.ware at ntlworld.com (alister) Date: Fri, 16 Jun 2017 11:03:47 GMT Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> <194b245e-f4dc-cc38-5a2d-b772574d2095@lucidity.plus.com> Message-ID: On Thu, 15 Jun 2017 21:17:05 +0100, Erik wrote: > On 15/06/17 15:10, Chris Angelico wrote: >> On Fri, Jun 16, 2017 at 12:00 AM, alister >> wrote: >>> Json is designed to be legal Javascript code & therefore directly >>> executable so no parser is posible. >>> >>> >> "no parser is possible"??? > > I *think* alister meant "so it is possible to not use a parser > [library]" (i.e., parse the stream using JavaScript's parser via eval() > - though I agree with everyone else who has said this should never be > done). > > I may be wrong about what alister meant, but the language reminds me of > a German colleague of mine from a few years back who wrote in some API > update documentation "Specifying parameter X is no longer an option". > What he meant was "is now mandatory" or "is no longer optional". To a > native English speaker, it reads as "can no longer be used" which is the > opposite of what he meant ... > > E. indeed, I missed a few words in my statement caused by me re-composing my thoughts on the fly. (The non native English speaker excuse is not an option for me) -- Man invented language to satisfy his deep need to complain. -- Lily Tomlin From alister.ware at ntlworld.com Fri Jun 16 07:12:05 2017 From: alister.ware at ntlworld.com (alister) Date: Fri, 16 Jun 2017 11:12:05 GMT Subject: Developers Who Use Spaces Make More Money! References: Message-ID: <92P0B.75271$7A4.14499@fx19.am4> On Thu, 15 Jun 2017 21:11:41 -0600, Ian Kelly wrote: > On Thu, Jun 15, 2017 at 8:51 PM, Larry Martell > wrote: >> On Thu, Jun 15, 2017 at 6:35 PM, Christopher Reimer >> wrote: >>> One commentator on a tech website admonished programmers for wasting >>> time by pressing the space bar four times instead of using tab. O_o >> >> Not in vi with >> >> set autoindent set expandtab > > Or in any halfway-decent editor with reasonable indentation settings. indeed if your editor can deal with something this simple i dread to think how much more time wasting will be caused by all of its other inefficiencies -- Instead, people would take pains to tell her that beauty was only skin-deep, as if a man ever fell for an attractive pair of kidneys. (Maskerade) From python.list at tim.thechases.com Fri Jun 16 07:40:28 2017 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 16 Jun 2017 06:40:28 -0500 Subject: Customise the virtualenv `activate` script In-Reply-To: <85h8zgv5pg.fsf@benfinney.id.au> References: <85mv9cwsy0.fsf@benfinney.id.au> <20170613075057.GA51505@cskk.homeip.net> <85h8zgv5pg.fsf@benfinney.id.au> Message-ID: <20170616064028.1eafa2e1@bigbox.christie.dr> On 2017-06-16 15:53, Ben Finney wrote: > > I must admit my initial preference would be the differently named > > wrapper. Surely users of the codebase will be invoking stuff via > > something opaque which sources the requisite things? > > That ?something opaque? is the ?$VENV/bin/activate? script; many > people who join the team will already know that, and I'm trying to > make use of that existing convention. > > > Actually, on trying to write something simple and flexible, since > > once made the venv is basicly state WRT the activate script, I'm > > leaning towards hacking the activate script, probably by keeping > > a distinct file off the the side and modifying activate to source > > it. > > Yeah, I'd much prefer to be told there's a hook to use, so that > someone who creates a standard Python virtualenv the conventional > way will not need to then hack that virtualenv. At least within virtualenvwrapper (I'm not sure whether they come with virtualenv proper), in my $WORKON_HOME and $WORKON_HOME/$VIRTUALENV/bin directories, I have a bunch of pre* and post* templates including preactivate and postactivate hooks in which I can put various bash scripting. I've only used them once or twice and don't have an example readily at hand, but it seems would give you what you're looking for. -tkc From gbs.deadeye at gmail.com Fri Jun 16 07:49:39 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=c3=bcller?=) Date: Fri, 16 Jun 2017 13:49:39 +0200 Subject: Standard lib version of something like enumerate() that takes a max count iteration parameter? In-Reply-To: References: <1497463783.4113781.1009502144.56C9EA51@webmail.messagingengine.com> <1497466402.492321.1009552520.29BE236F@webmail.messagingengine.com> Message-ID: <878db1f1-0343-8873-19ef-47ab7a0b54bc@gmail.com> Am 15.06.2017 um 07:09 schrieb Jussi Piitulainen: > Andre M?ller writes: > >> I'm a fan of infinite sequences. Try out itertools.islice. >> You should not underestimate this very important module. >> >> Please read also the documentation: >> https://docs.python.org/3.6/library/itertools.html >> >> from itertools import islice >> >> iterable = range(10000000000) >> # since Python 3 range is a lazy evaluated object >> # using this just as a dummy >> # if you're using legacy Python (2.x), then use the xrange function for it >> # or you'll get a memory error >> >> max_count = 10 >> step = 1 >> >> for i, element in enumerate(islice(iterable, 0, max_count, step), start=1): >> print(i, element) > I like to test this kind of thing with iter("abracadabra") and look at > the remaining elements, just to be sure that they are still there. > > from itertools import islice > > s = iter("abracadabra") > for i, element in enumerate(islice(s, 3)): > print(i, element) > > print(''.join(s)) > > Prints this: > > 0 a > 1 b > 2 r > acadabra > > One can do a similar check with iter(range(1000)). The range object > itself does not change when its elements are accessed. Very interesting. Normally this should not work. The behavior is unexpected. So you taught me, what can happen. Thank You :-) Normally you don't see very often iter(). If you've short sequences which are str, you can just use index access. My example is for something, which is bigger than memory. Otherwise you've sometimes objects which doesn't support index access like sets or generators. Then you can use this nifty trick. Instead of using: s = iter('abracadabra') # no direct access to the str object You should use: s = 'abracadabra' # direct access to object iterator = iter(s) # makes an iterator which is accessing s. The str object does not change. # s is still 'abracadabra' # also you can make more than one iterator of the same object iterator2 = iter(s) iterator3 = iter(s) iterator4 = iter(s) # they only rely on s, but they are independent iterators # s won't change Another trick is: # incomplete chunks are left out list(zip(*[iter(s)]*4)) # -> [('a', 'b', 'r', 'a'), ('c', 'a', 'd', 'a')] # incomplete chunks will have None in the list list(itertools.zip_longest(*[iter(s)]*4)) # -> [('a', 'b', 'r', 'a'), ('c', 'a', 'd', 'a'), ('b', 'r', 'a', None)] # to impress your friends you can do for chunk in itertools.zip_longest(*[iter(s)]*4): chunked_str = ''.join(c for c in chunk if c) # generator expression inside join with condition print(chunked_str) It took long time for me to understand this. Someone has written this nifty tick in the python irc channel. You should create every time a new iterator with iter, when iterating over something again. In your example you're iterating twice over the same iterator. When you're using for example a for loop, it creates internally an iterator of your object, if it's supported. It gets element by element and assigns it to one_char. # creates iterator for s and iterates over it for one_char in s: print(one_char) # creates again a new iterator for s and iterates over it for one_char in s: print(one_char) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From __peter__ at web.de Fri Jun 16 08:15:26 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 16 Jun 2017 14:15:26 +0200 Subject: Standard lib version of something like enumerate() that takes a max count iteration parameter? References: <1497463783.4113781.1009502144.56C9EA51@webmail.messagingengine.com> <1497466402.492321.1009552520.29BE236F@webmail.messagingengine.com> <878db1f1-0343-8873-19ef-47ab7a0b54bc@gmail.com> Message-ID: Andre M?ller wrote: > # to impress your friends you can do > for chunk in itertools.zip_longest(*[iter(s)]*4): > chunked_str = ''.join(c for c in chunk if c) # generator expression > inside join with condition > print(chunked_str) This can be simplified with a fillvalue >>> s = "abracadabra" >>> for chunk in itertools.zip_longest(*[iter(s)]*4, fillvalue=""): ... print("".join(chunk)) ... abra cada bra but for sequences I prefer regular slicing: >>> N = 4 >>> for start in range(0, len(s), N): ... print(s[start:start+N]) ... abra cada bra From mal at europython.eu Fri Jun 16 09:06:25 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Fri, 16 Jun 2017 15:06:25 +0200 Subject: EuroPython 2017: Django Girls Workshop Message-ID: <46eb9821-0dd8-83a4-51ff-8691efcbac0e@europython.eu> EuroPython 2017 is happy to host and sponsor a Django Girls free workshop on Sunday 9th, from 9:00 to 18:00 CEST. The non-profit organization FuzzyBrains will lead you through HTML/CSS, Python/Django to the design of a new blog in a single day workshop. No prior programming knowledge is needed to participate! If you would like to take part, hurry up and apply for a spot on the Django Girls website. * Django Girls EuroPython 2017 * https://ep2017.europython.eu/en/events/django-girls-workshop/ EuroPython 2017 sponsors the event, providing room, catering and free conference tickets for the trainers, together with other facilities. If you?d like to know more about Django Girls, please check their website or their social network coordinates on Facebook and Twitter. EuroPython ticket sales are picking up -------------------------------------- If you want to join the EuroPython fun for the whole week, be sure to get your tickets as soon as possible, since ticket sales have picked up quite a bit after we announced the schedule. Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/875699933269176320 Thanks. From perlovaa at gmail.com Fri Jun 16 09:17:15 2017 From: perlovaa at gmail.com (Alla Perlova) Date: Fri, 16 Jun 2017 06:17:15 -0700 (PDT) Subject: How to calculate Value in (%) and than create new column with percentage of used time. (Python) Message-ID: <00f57d8a-44a6-46c0-a539-565437ed6ec7@googlegroups.com> Dear folks, I need some help with: I am working with some Hospital data, and I need to calculate Operating Room Utilization per day df.groupby(by = ['Actual_Surgery_Day','Actual_Surgery_Room']) .agg({'Actual_Surgery_Duratation' : 'sum', 'Procedure_Code' : 'count'}) Actual_Surgery_Duratation Procedure_Code 609.0 10 561.0 7 704.0 5 Actual_Surgery_Day Actual_Surgery_Room 20001 20002 20003 .......... 20016 2016-01-08 So, I need to calculate a simple following equation **Z=960-x-18(y-1)** 960 min = optimal time (in min) Room Utilization per day x = Time for all procedures per Day; column - (Actual_Surgery_Duratation) y = Procedure (amount per day in each Room (- 1), first case); column - (Procedure_Code) 18 min = break time between each procedure How do I calculate Room Utilization in %? **Example**,first row table: z=960-x-18(10-1) z=960 min (wanted time) - 609 min (actual time) - 162 (breaktime) z=189 min time not used What is the percentage of used time? in % (960 min - 189 min) I appreciate any help. Alla From grant.b.edwards at gmail.com Fri Jun 16 10:29:41 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 16 Jun 2017 14:29:41 +0000 (UTC) Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> <85d1a4v5k0.fsf@benfinney.id.au> Message-ID: On 2017-06-16, Ben Finney wrote: > alister writes: > >> Json is designed to be legal Javascript code & therefore directly >> executable so no parser is posible. > > JSON is designed to be *a strictly limited subset* of legal JavaScript > that only defines data structures. The explicit goal is that it is > statically parseable as non-executable data. That doesn't mean that it's reasonable/acceptable practice to eval() a string from an untrusted source because it _might_ be JSON. -- Grant Edwards grant.b.edwards Yow! I brought my BOWLING at BALL -- and some DRUGS!! gmail.com From ben+python at benfinney.id.au Fri Jun 16 10:37:27 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 17 Jun 2017 00:37:27 +1000 Subject: [OT] is JSON all that great? - was Re: API Help References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> <85d1a4v5k0.fsf@benfinney.id.au> Message-ID: <858tksuhg8.fsf@benfinney.id.au> Grant Edwards writes: > On 2017-06-16, Ben Finney wrote: > > JSON is designed to be *a strictly limited subset* of legal > > JavaScript that only defines data structures. The explicit goal is > > that it is statically parseable as non-executable data. > > That doesn't mean that it's reasonable/acceptable practice to eval() a > string from an untrusted source because it _might_ be JSON. Yes. We appear to be in firm agreement. -- \ ?It is always a silly thing to give advice, but to give good | `\ advice is absolutely fatal.? ?Oscar Wilde, _The Portrait of Mr. | _o__) W. H._, 1889-07 | Ben Finney From toby at tobiah.org Fri Jun 16 11:48:32 2017 From: toby at tobiah.org (Tobiah) Date: Fri, 16 Jun 2017 08:48:32 -0700 Subject: API Help References: <71944145-3b81-263d-25d1-99f211f1c467@lucidity.plus.com> <53beeec5-6ff1-7ab2-2290-1f8604522246@lucidity.plus.com> Message-ID: > I still think it _could_ be the output of a Python repr() or similar > (something that is expected to be evaluated as a Python expression). It may be valid fodder for python eval(), but if it came from repr() It would have used single quotes, yes? From tjreedy at udel.edu Fri Jun 16 12:28:54 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 16 Jun 2017 12:28:54 -0400 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months Message-ID: https://thenewstack.io/instagram-makes-smooth-move-python-3/ -- Terry Jan Reedy From rauklei at gmail.com Fri Jun 16 13:43:23 2017 From: rauklei at gmail.com (=?UTF-8?B?UmF1a2xlaSBQLiBTLiBHdWltYXLDo2Vz?=) Date: Fri, 16 Jun 2017 17:43:23 +0000 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months In-Reply-To: References: Message-ID: Very Nice. Em sex, 16 de jun de 2017 ?s 13:30, Terry Reedy escreveu: > https://thenewstack.io/instagram-makes-smooth-move-python-3/ > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list > From paul.james.barry at gmail.com Fri Jun 16 13:56:47 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Fri, 16 Jun 2017 18:56:47 +0100 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months In-Reply-To: References: Message-ID: The process they followed is discussed in their recent Keynote at PyCon 2017: https://youtu.be/66XoCk79kjM Well worth the 40 minutes it takes to watch :-) Paul. On 16 June 2017 at 18:43, Rauklei P. S. Guimar?es wrote: > Very Nice. > > Em sex, 16 de jun de 2017 ?s 13:30, Terry Reedy > escreveu: > > > https://thenewstack.io/instagram-makes-smooth-move-python-3/ > > -- > > Terry Jan Reedy > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From tjol at tjol.eu Fri Jun 16 14:16:24 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 16 Jun 2017 20:16:24 +0200 Subject: sqlite in 2.7 on redhat 6 In-Reply-To: References: Message-ID: <59442078$0$1565$e4fe514c@news.kpn.nl> On 15/06/17 14:45, Larry Martell wrote: > I am trying to use sqlite > > $ python2.7 > Python 2.7.10 (default, Feb 22 2016, 12:13:36) > [GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import _sqlite3 > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named _sqlite3 > > It's there at: /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/_sqlite3.so > but that is not in my path This looks like it's installed for an SCL (https://www.softwarecollections.org/en/scls/rhscl/python27/). Are you running the SCL python, or some other python? You either have to run the SCL python (My guess without having an EL6 system lying around: scl enable python27) or install sqlite in whatever python installation you're actually using. . > > I tried adding /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/ > to my path and then it fails with: > > ImportError: libpython2.7.so.1.0: cannot open shared object file: No > such file or directory > > Isn't sqlite part of the standard lib? Shouldn't this just work? > From saxri89 at gmail.com Fri Jun 16 15:58:36 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Fri, 16 Jun 2017 12:58:36 -0700 (PDT) Subject: python and post gis question Message-ID: <582d3128-d44f-4f99-8e56-c777bb00d83b@googlegroups.com> I have create a python script where use post gis queries to automate some intersection tasks using postgis database. in my database I have polygons,points and lines. here my snippet code of my script : try: if str(geomtype) == 'point': geomtype = 1 elif str(geomtype) == 'linestring': geomtype = 2 elif str(geomtype) == 'polygon': geomtype = 3 else: raise TypeError() sql = "\nDROP TABLE IF EXISTS {0}.{1};\nCREATE TABLE {0}.{1} AS (SELECT {4},\n(st_dump(ST_CollectionExtract(st_intersection(dbgis.{2}.shape,dbgis.{3}.shape)::GEOMETRY(GEOMETRY,2345),{5}))).geom AS shape\nFROM dbgis.{2}, dbgis.{3} WHERE st_intersects(dbgis.{2}.shape,dbgis.{3}.shape) AND {5}=3);\nCREATE INDEX idx_{2}_{3}_{6} ON {0}.{1} USING GIST (shape);\nALTER TABLE {0}.{1} ADD COLUMN id SERIAL;\nALTER TABLE {0}.{1} ADD COLUMN my_area double precision;\nUPDATE {0}.{1} SET my_area = ST_AREA(shape::GEOMETRY);\nALTER TABLE {0}.{1} ADD PRIMARY KEY (id);\nSELECT pop_g_col('{0}.{1}'::REGCLASS);\nGRANT ALL ON {0}.{1} TO GROUP u_cl;\nGRANT ALL ON {0}.{1} TO GROUP my_user;\n-- VACUUM ANALYZE {0}.{1};\n".format(schema, table, tablea, tableb, selects, geomtype, id_gen()) return sql this post gis sql query work nice but I need the new field where I want to add my_area to create only for polygons layers. that code create for all layers (lines,points,polygons) that field my_area if layer is point or line then take value 0 I don't like that I don't need it. how to change this code to create my_area only in polygons ? From fapb88ve at gmail.com Fri Jun 16 18:32:36 2017 From: fapb88ve at gmail.com (Frank Pinto) Date: Fri, 16 Jun 2017 18:32:36 -0400 Subject: Google Sheets API Error Message-ID: Hello, I'm building an app that needs to do the following things: a) checks a Google Sheet for email and name of a client, b) logs into a dropbox account and downloads some files (only those files with the name from the google sheet), c) does some work on the files d) sends the new files to the clients with the information from the google sheets. It needs to do this from the first moment it runs till 7:30 PM local time (using a while loop). The problem I'm having is that when it starts running, the first iteration runs fine. When it gets to the second, the google sheets function I created that accesses the data of the sheets does not work (even though it did the first time). The console yields the following error: "FileNotFoundError: [Errno 2] No such file or directory: 'client_id.json'" This is weird, because the json file obviously is not moved in between runs. Here's an overview of the structure of the code: def full(): global logo, plogo, plogow, plogoh, client, metadata, logo_path inst = 1 logo_path = os.path.abspath('.\\logo\\p4.png') client = dropbox.client.DropboxClient(XXXXX') while datetime.datetime.now().time() <= datetime.time(19, 45): if inst == 1: custo = customers() if len(custo) == 0: pass else: metadata = client.metadata('/') d_files = files() send_f = pd.merge(d_files, custo, left_on='c_name', right_on='Nombre', how='inner') dl_files(send_f, custo) else: temp = customers() send = temp[~temp.isin(custo)].dropna() custo = customers() if len(send) == 0: pass else: metadata = client.metadata('/') d_files = files() send_f = pd.merge(d_files, custo, left_on='c_name', right_on='Nombre', how='inner') dl_files(send_f, send) time.sleep(30) inst += 1 The 'customers()' function that is giving me problems: def customers(): scope = ['https://spreadsheets.google.com/feeds'] creds = ServiceAccountCredentials.from_json_keyfile_name('client_id.json', scope) client = gspread.authorize(creds) sheet = client.open('RC').sheet1 c_list = sheet.get_all_records() df = pd.DataFrame(c_list) return df I really have no idea why this is happening. Does anybody might have a clue? Thank you, -- Frank Pinto From python at lucidity.plus.com Fri Jun 16 19:11:48 2017 From: python at lucidity.plus.com (Erik) Date: Sat, 17 Jun 2017 00:11:48 +0100 Subject: [OT] is JSON all that great? - was Re: API Help In-Reply-To: References: <34b2b105-a3be-ae3c-72f4-b167ee7726b4@lucidity.plus.com> <2bb3773e-1d88-7b91-db6d-dc5dffc0de98@kynesim.co.uk> <194b245e-f4dc-cc38-5a2d-b772574d2095@lucidity.plus.com> Message-ID: <3fedae35-0243-1b83-d9b3-134835e518e3@lucidity.plus.com> On 16/06/17 12:03, alister wrote: > (The non native English speaker excuse is not an option for me) Does that mean it's mandatory for you? I'm confused :D :D E. From cs at zip.com.au Fri Jun 16 19:39:45 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 17 Jun 2017 09:39:45 +1000 Subject: Customise the virtualenv `activate` script In-Reply-To: <20170616064028.1eafa2e1@bigbox.christie.dr> References: <20170616064028.1eafa2e1@bigbox.christie.dr> Message-ID: <20170616233945.GA91832@cskk.homeip.net> On 16Jun2017 06:40, Tim Chase wrote: >At least within virtualenvwrapper (I'm not sure whether they come >with virtualenv proper), in my $WORKON_HOME and >$WORKON_HOME/$VIRTUALENV/bin directories, I have a bunch of pre* and >post* templates including preactivate and postactivate hooks in which >I can put various bash scripting. I've only used them once or twice >and don't have an example readily at hand, but it seems would give you >what you're looking for. And just for added data, I've just inspected the 'activate" script in my 3.6.1 virtualenv (which looks like it was made using virtualenv-3.5) and it has no hooks in the shell code, so Tim's stuff looks like it comes from virtualenvwrapper. Cheers, Cameron Simpson From mcmxl at hotmail.com Sat Jun 17 01:04:15 2017 From: mcmxl at hotmail.com (mcmxl at hotmail.com) Date: Fri, 16 Jun 2017 22:04:15 -0700 (PDT) Subject: How to calculate Value in (%) and than create new column with percentage of used time. (Python) In-Reply-To: <00f57d8a-44a6-46c0-a539-565437ed6ec7@googlegroups.com> References: <00f57d8a-44a6-46c0-a539-565437ed6ec7@googlegroups.com> Message-ID: <48bdfb40-7a4a-4234-adf5-a9a0587957bf@googlegroups.com> I assigned the optimal time of 960min to x. >>> x=960.0 Then I assigned the time used 189min to y. >>> y=189.0 Then I divide the time used(y) by the optimal time(x) and multiply the answer by 100 and assign the answer to z. >>> z=(y/x)*100 The answer. >>> z 19.6875 For Python to give you an accurate answer you must include decimal points. I hope this helps. From saxri89 at gmail.com Sat Jun 17 08:20:36 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sat, 17 Jun 2017 05:20:36 -0700 (PDT) Subject: python and post gis question In-Reply-To: <582d3128-d44f-4f99-8e56-c777bb00d83b@googlegroups.com> References: <582d3128-d44f-4f99-8e56-c777bb00d83b@googlegroups.com> Message-ID: <3e103ad2-1e4d-420a-9777-9133d4e72f5a@googlegroups.com> ?? ?????????, 16 ??????? 2017 - 10:59:10 ?.?. UTC+3, ? ??????? Xristos Xristoou ??????: > I have create a python script where use post gis queries to automate some intersection tasks using postgis database. > > in my database I have polygons,points and lines. > > here my snippet code of my script : > > try: > if str(geomtype) == 'point': > geomtype = 1 > elif str(geomtype) == 'linestring': > geomtype = 2 > elif str(geomtype) == 'polygon': > geomtype = 3 > else: > raise TypeError() > sql = "\nDROP TABLE IF EXISTS {0}.{1};\nCREATE TABLE {0}.{1} AS (SELECT {4},\n(st_dump(ST_CollectionExtract(st_intersection(dbgis.{2}.shape,dbgis.{3}.shape)::GEOMETRY(GEOMETRY,2345),{5}))).geom AS shape\nFROM dbgis.{2}, dbgis.{3} WHERE st_intersects(dbgis.{2}.shape,dbgis.{3}.shape) AND {5}=3);\nCREATE INDEX idx_{2}_{3}_{6} ON {0}.{1} USING GIST (shape);\nALTER TABLE {0}.{1} ADD COLUMN id SERIAL;\nALTER TABLE {0}.{1} ADD COLUMN my_area double precision;\nUPDATE {0}.{1} SET my_area = ST_AREA(shape::GEOMETRY);\nALTER TABLE {0}.{1} ADD PRIMARY KEY (id);\nSELECT pop_g_col('{0}.{1}'::REGCLASS);\nGRANT ALL ON {0}.{1} TO GROUP u_cl;\nGRANT ALL ON {0}.{1} TO GROUP my_user;\n-- VACUUM ANALYZE {0}.{1};\n".format(schema, table, tablea, tableb, selects, geomtype, id_gen()) > return sql > this post gis sql query work nice but I need the new field where I want to add my_area to create only for polygons layers. > > that code create for all layers (lines,points,polygons) that field my_area if layer is point or line then take value 0 I don't like that I don't need it. > > how to change this code to create my_area only in polygons ? what you mean where? From tjreedy at udel.edu Sat Jun 17 13:21:48 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 17 Jun 2017 13:21:48 -0400 Subject: How to calculate Value in (%) and than create new column with percentage of used time. (Python) In-Reply-To: <48bdfb40-7a4a-4234-adf5-a9a0587957bf@googlegroups.com> References: <00f57d8a-44a6-46c0-a539-565437ed6ec7@googlegroups.com> <48bdfb40-7a4a-4234-adf5-a9a0587957bf@googlegroups.com> Message-ID: On 6/17/2017 1:04 AM, mcmxl at hotmail.com wrote: > I assigned the optimal time of 960min to x. >>>> x=960.0 > > Then I assigned the time used 189min to y. >>>> y=189.0 > > Then I divide the time used(y) by the optimal time(x) and multiply the answer by 100 and assign the answer to z. >>>> z=(y/x)*100 > > The answer. >>>> z > 19.6875 > > For Python to give you an accurate answer you must include decimal points. > I hope this helps. In Python 3, they do not matter here. >>> (189/960)*100 19.6875 In 2.7: >>> (189/960)*100 0 >>> from __future__ import division >>> (189/960)*100 19.6875 -- Terry Jan Reedy From ramsinghdayal at gmail.com Sat Jun 17 15:03:18 2017 From: ramsinghdayal at gmail.com (Ramdayal Singh) Date: Sat, 17 Jun 2017 12:03:18 -0700 (PDT) Subject: Basic: scikits.rsformats Message-ID: <8d5eb43c-062b-444e-97fb-aece6e00ff6b@googlegroups.com> How can I know what are the applications of scikits.rsformats? From m at funkyhat.org Sat Jun 17 18:49:07 2017 From: m at funkyhat.org (Matt Wheeler) Date: Sat, 17 Jun 2017 22:49:07 +0000 Subject: Google Sheets API Error In-Reply-To: References: Message-ID: On Fri, Jun 16, 2017, 23:32 Frank Pinto wrote: > I'm building an app that needs to do the following things: a) checks a > Google Sheet for email and name of a client, b) logs into a dropbox account > and downloads some files (only those files with the name from the google > sheet), c) does some work on the files d) sends the new files to the > clients with the information from the google sheets. It needs to do this > from the first moment it runs till 7:30 PM local time (using a while loop). > > The problem I'm having is that when it starts running, the first iteration > runs fine. When it gets to the second, the google sheets function I created > that accesses the data of the sheets does not work (even though it did the > first time). The console yields the following error: > > "FileNotFoundError: [Errno 2] No such file or directory: 'client_id.json'" > My best guess is some of your code which you haven't shown us is calling `os.chdir()` and then you're not moving back. When I encounter this problem I usually mitigate it using a context manager something like this: @contextlib.contextmanager def pushd(dir): prev = os.getcwd() os.chdir(dir) yield os.chdir(prev) And then with pushd(dir): Stuff in the dir This is weird, because the json file obviously is not moved in between > runs. Here's an overview of the structure of the code: But it is just a guess, as your intention has been mangled somewhere along the way, and in any case the code seems to be missing parts. -- -- Matt Wheeler http://funkyh.at From nad at python.org Sat Jun 17 22:31:05 2017 From: nad at python.org (Ned Deily) Date: Sat, 17 Jun 2017 22:31:05 -0400 Subject: [RELEASE] Python 3.6.2rc1 is now available for testing Message-ID: On behalf of the Python development community and the Python 3.6 release team, I would like to announce the availability of Python 3.6.2rc1. 3.6.2rc1 is the first release candidate for Python 3.6.2, the next maintenance release of Python 3.6. While 3.6.2rc1 is a preview release and, thus, not intended for production environments, we encourage you to explore it and provide feedback via the Python bug tracker (https://bugs.python.org). Please see "What?s New In Python 3.6" for more information: https://docs.python.org/3.6/whatsnew/3.6.html You can find Python 3.6.2rc1 here: https://www.python.org/downloads/release/python-362rc1/ and its change log here: https://docs.python.org/3.6/whatsnew/changelog.html#python-3-6-2-release-candidate-1 3.6.2 is planned for final release on 2017-06-30 with the next maintenance release expected to follow in about 3 months. More information about the 3.6 release schedule can be found here: https://www.python.org/dev/peps/pep-0494/ -- Ned Deily nad at python.org -- [] From tjreedy at udel.edu Sat Jun 17 23:48:58 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 17 Jun 2017 23:48:58 -0400 Subject: [RELEASE] Python 3.6.2rc1 is now available for testing In-Reply-To: References: Message-ID: On 6/17/2017 10:31 PM, Ned Deily wrote: > You can find Python 3.6.2rc1 here: > > https://www.python.org/downloads/release/python-362rc1/ Running Debug|Win32 interpreter... == CPython 3.6.1+ (heads/3.6:0a794a3256, Jun 17 2017, 17:03:32) has 1 error, winreg hangs, resists Keyboard interrupt, must click [X] to close. Windows amd64 installer. has 5 more errors. test_codecencodings_iso2022 multiple failures test_random multiple failures test_sax multiple failures test_regrtest failed in test_pcbuild_rt, FileNotFoundError in subprocess test_tools 2to3 it seems test_winreg looped forever, keyboard interrupt ignored for a minute, then locked console in threading waiting or lock or lock acquire. I reran, redownloaded and reinstalled, same thing. -- Terry Jan Reedy From greg.ewing at canterbury.ac.nz Sun Jun 18 01:25:59 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 18 Jun 2017 17:25:59 +1200 Subject: Google Sheets API Error In-Reply-To: References: Message-ID: Matt Wheeler wrote: > My best guess is some of your code which you haven't shown us is calling > `os.chdir()` and then you're not moving back. > > When I encounter this problem I usually mitigate it using a context manager Or avoid using chdir at all, and use full pathnames to refer to things in other directories. -- Greg From saxri89 at gmail.com Sun Jun 18 03:46:32 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sun, 18 Jun 2017 00:46:32 -0700 (PDT) Subject: python and post gis question In-Reply-To: <582d3128-d44f-4f99-8e56-c777bb00d83b@googlegroups.com> References: <582d3128-d44f-4f99-8e56-c777bb00d83b@googlegroups.com> Message-ID: ?? ?????????, 16 ??????? 2017 - 10:59:10 ?.?. UTC+3, ? ??????? Xristos Xristoou ??????: > I have create a python script where use post gis queries to automate some intersection tasks using postgis database. > > in my database I have polygons,points and lines. > > here my snippet code of my script : > > try: > if str(geomtype) == 'point': > geomtype = 1 > elif str(geomtype) == 'linestring': > geomtype = 2 > elif str(geomtype) == 'polygon': > geomtype = 3 > else: > raise TypeError() > sql = "\nDROP TABLE IF EXISTS {0}.{1};\nCREATE TABLE {0}.{1} AS (SELECT {4},\n(st_dump(ST_CollectionExtract(st_intersection(dbgis.{2}.shape,dbgis.{3}.shape)::GEOMETRY(GEOMETRY,2345),{5}))).geom AS shape\nFROM dbgis.{2}, dbgis.{3} WHERE st_intersects(dbgis.{2}.shape,dbgis.{3}.shape) AND {5}=3);\nCREATE INDEX idx_{2}_{3}_{6} ON {0}.{1} USING GIST (shape);\nALTER TABLE {0}.{1} ADD COLUMN id SERIAL;\nALTER TABLE {0}.{1} ADD COLUMN my_area double precision;\nUPDATE {0}.{1} SET my_area = ST_AREA(shape::GEOMETRY);\nALTER TABLE {0}.{1} ADD PRIMARY KEY (id);\nSELECT pop_g_col('{0}.{1}'::REGCLASS);\nGRANT ALL ON {0}.{1} TO GROUP u_cl;\nGRANT ALL ON {0}.{1} TO GROUP my_user;\n-- VACUUM ANALYZE {0}.{1};\n".format(schema, table, tablea, tableb, selects, geomtype, id_gen()) > return sql > this post gis sql query work nice but I need the new field where I want to add my_area to create only for polygons layers. > > that code create for all layers (lines,points,polygons) that field my_area if layer is point or line then take value 0 I don't like that I don't need it. > > how to change this code to create my_area only in polygons ? i wabt to ccreate a new field only in polygons layera From saxri89 at gmail.com Sun Jun 18 14:49:59 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sun, 18 Jun 2017 11:49:59 -0700 (PDT) Subject: json to access using python Message-ID: <7e8db168-8846-4a66-bb57-dc3acbb14619@googlegroups.com> hello I have a json url and i want from this url to update my table in microsoft access,how to do that using python or some tool of microsoft access ? From pavlovevidence at gmail.com Sun Jun 18 15:27:52 2017 From: pavlovevidence at gmail.com (pavlovevidence at gmail.com) Date: Sun, 18 Jun 2017 12:27:52 -0700 (PDT) Subject: Reciprocal data structures Message-ID: I'm not sure if "reciprocal" is the right word, or if there is an official term for this. I am thinking of a list that actively maintains in its items a member that contains the item's own index in the list. Basically, the item knows its index into the list and the list ensures that the index remains in sync. If the list is changed, the list updates the indices. This is a the that might be useful in implementing observer pattern. I guess I am questioning if there's a well-known package that does anything like this? I don't know what the official computer science term might be for this so I'm having trouble googling for it. I don't need this kind of thing a lot, but to me it has the feel of something that should be already available somewhere. Carl Banks From breamoreboy at gmail.com Sun Jun 18 15:34:14 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 18 Jun 2017 12:34:14 -0700 (PDT) Subject: json to access using python In-Reply-To: <7e8db168-8846-4a66-bb57-dc3acbb14619@googlegroups.com> References: <7e8db168-8846-4a66-bb57-dc3acbb14619@googlegroups.com> Message-ID: <91da668c-e463-4bc1-b650-2cfb6b973f03@googlegroups.com> On Sunday, June 18, 2017 at 7:50:10 PM UTC+1, Xristos Xristoou wrote: > hello > > > I have a json url and i want from this url to update my table in microsoft access,how to do that using python or some tool of microsoft access ? You need to do some research first. Then you run an editor and type some code in. Then you run the code. If it does not work and you do not understand why you then ask a question here. You might like to reference this http://www.catb.org/esr/faqs/smart-questions.html and this http://sscce.org/ first. Kindest regards. Mark Lawrence. From rosuav at gmail.com Sun Jun 18 16:04:57 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 19 Jun 2017 06:04:57 +1000 Subject: Reciprocal data structures In-Reply-To: References: Message-ID: On Mon, Jun 19, 2017 at 5:27 AM, wrote: > I am thinking of a list that actively maintains in its items a member that contains the item's own index in the list. Basically, the item knows its index into the list and the list ensures that the index remains in sync. If the list is changed, the list updates the indices. This is a the that might be useful in implementing observer pattern. > With a list? No, I would say it's a bad idea. But with a dictionary, you certainly can. Consider something that loads up something based on its unique ID (which could be a number or a string or something), and then the thing itself has that ID. Very common in databasing, for instance. You can then have a lookup dictionary where you grab them from your cache, and the objects themselves have a "foo.id" attribute. The IDs are immutable and eternal, unlike list indices, and they are truly a feature of the object, so this makes a lot of sense. And it's something I've done reasonably often. ChrisA From none at invalid.com Sun Jun 18 17:18:29 2017 From: none at invalid.com (mm0fmf) Date: Sun, 18 Jun 2017 22:18:29 +0100 Subject: json to access using python In-Reply-To: <91da668c-e463-4bc1-b650-2cfb6b973f03@googlegroups.com> References: <7e8db168-8846-4a66-bb57-dc3acbb14619@googlegroups.com> <91da668c-e463-4bc1-b650-2cfb6b973f03@googlegroups.com> Message-ID: On 18/06/2017 20:34, breamoreboy at gmail.com wrote: > On Sunday, June 18, 2017 at 7:50:10 PM UTC+1, Xristos Xristoou wrote: >> hello >> >> >> I have a json url and i want from this url to update my table in microsoft access,how to do that using python or some tool of microsoft access ? > > You need to do some research first. Then you run an editor and type some code in. Then you run the code. If it does not work and you do not understand why you then ask a question here. You might like to reference this http://www.catb.org/esr/faqs/smart-questions.html and this http://sscce.org/ first. > > Kindest regards. > > Mark Lawrence. > Mark, I thought this guy was nymshift of Ferrous Cranium or whatever he was called. From valkrem at yahoo.com Sun Jun 18 17:30:52 2017 From: valkrem at yahoo.com (Val Krem) Date: Sun, 18 Jun 2017 21:30:52 +0000 (UTC) Subject: course References: <1103357550.1193469.1497821452041.ref@mail.yahoo.com> Message-ID: <1103357550.1193469.1497821452041@mail.yahoo.com> Hi all, Is there on line course in Python? I am looking for a level between beginner and intermediate. I would appreciate if you could suggest me? Thank you. From no.email at nospam.invalid Sun Jun 18 20:20:28 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 18 Jun 2017 17:20:28 -0700 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87poe03k1f.fsf@nightsong.com> I always thought the GIL removal obstacle was the need to put locks around every refcount adjustment, and the only real cure for that is to use a tracing GC. That is a good idea in many ways, but it would break the existing C API quite seriously. Reworking the C modules in the stdlib would be a large but not impossible undertaking. The many external C modules out there would be more of an issue. From mradul.k.dhakad at gmail.com Sun Jun 18 22:57:22 2017 From: mradul.k.dhakad at gmail.com (mradul dhakad) Date: Sun, 18 Jun 2017 22:57:22 -0400 Subject: Error while connecting Teradata Message-ID: Hi All , I am getting below Error while connecting Teradata using Python: - File "C:\Python27\Lib\site-packages\teradata\udaexec.py", line 183, in connect **args)) File "C:\Python27\Lib\site-packages\teradata\tdodbc.py", line 427, in __init__ connectParams["DRIVER"] = determineDriver(dbType, driver) File "C:\Python27\Lib\site-packages\teradata\tdodbc.py", line 391, in determineDriver "Available drivers: {}".format(dbType, ",".join(drivers))) InterfaceError: ('DRIVER_NOT_FOUND', "No driver found for 'Teradata' I am using below code to connect : - import teradata udaExec = teradata.UdaExec (appName="HelloWorld", version="1.0", logConsole=False) session = udaExec.connect(method="odbc", system="tdpr101", username="xxx", password="xxx"); Could anyone please let me know how to resolve this Error. Thanks, Mradul From no.email at nospam.invalid Sun Jun 18 23:57:56 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 18 Jun 2017 20:57:56 -0700 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months References: Message-ID: <8760fsk4sb.fsf@nightsong.com> Paul Barry writes: > The process they followed is discussed in their recent Keynote at PyCon > 2017: https://youtu.be/66XoCk79kjM > Well worth the 40 minutes it takes to watch :-) If it takes 40 minutes to describe how they did it, that sounds like more hassle than most users of working py2 code probably want to deal with. From steve at pearwood.info Mon Jun 19 01:52:07 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 19 Jun 2017 05:52:07 GMT Subject: Instagram: 40% Py3 to 99% Py3 in 10 months References: <8760fsk4sb.fsf@nightsong.com> Message-ID: <59476687$0$1619$c3e8da3$5496439d@news.astraweb.com> On Sun, 18 Jun 2017 20:57:56 -0700, Paul Rubin wrote: > Paul Barry writes: >> The process they followed is discussed in their recent Keynote at PyCon >> 2017: https://youtu.be/66XoCk79kjM Well worth the 40 minutes it takes >> to watch :-) > > If it takes 40 minutes to describe how they did it, that sounds like > more hassle than most users of working py2 code probably want to deal > with. Well that depends on whether you have 100 lines of code or 100 million lines of code. One of the guys in my office migrated the Python code base for one of our internal projects overnight. The cheeky bugger didn't even tell us he was going to do it, he just stayed in the office one night for an extra three or four hours and presented us with a fait-accompli the next day. Admittedly it wasn't a huge code base, by memory something like twenty scripts and modules and 5-10 KLOC, give or take. That's not bad. I've spent an hour just formatting the comments in a single module *wink* -- Steve From steve at pearwood.info Mon Jun 19 01:54:49 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 19 Jun 2017 05:54:49 GMT Subject: Reciprocal data structures References: Message-ID: <59476729$0$1619$c3e8da3$5496439d@news.astraweb.com> On Mon, 19 Jun 2017 06:04:57 +1000, Chris Angelico wrote: > On Mon, Jun 19, 2017 at 5:27 AM, wrote: >> I am thinking of a list that actively maintains in its items a member >> that contains the item's own index in the list. Basically, the item >> knows its index into the list and the list ensures that the index >> remains in sync. If the list is changed, the list updates the indices. >> This is a the that might be useful in implementing observer pattern. How would you use this? Can you give some demonstration pseudo-code? > With a list? No, I would say it's a bad idea. Why a bad idea? As opposed to "can't be done", or "too hard and slow". > But with a dictionary, you > certainly can. Consider something that loads up something based on its > unique ID (which could be a number or a string or something), and then > the thing itself has that ID. Very common in databasing, for instance. > You can then have a lookup dictionary where you grab them from your > cache, and the objects themselves have a "foo.id" attribute. > The IDs are immutable and eternal, unlike list indices, and they are > truly a feature of the object, so this makes a lot of sense. And it's > something I've done reasonably often. -- Steve From rosuav at gmail.com Mon Jun 19 02:02:45 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 19 Jun 2017 16:02:45 +1000 Subject: Reciprocal data structures In-Reply-To: <59476729$0$1619$c3e8da3$5496439d@news.astraweb.com> References: <59476729$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Jun 19, 2017 at 3:54 PM, Steven D'Aprano wrote: >> With a list? No, I would say it's a bad idea. > > > Why a bad idea? > > As opposed to "can't be done", or "too hard and slow". Maintaining a record of list indices inside an object, with the specific proviso that: > If the list is changed, the list updates the indices. ? I think it's a bad idea because it's trying to use a changeable position as an attribute of an object. It seems like a job that's much better done with an immutable identifier. I might do it as the OP suggested *if* the IDs are never reused or deleted (or if you delete by replacing the object with None or something), which would make indices effectively immutable. But the idea that you should have list mutations that have to go and update everything else? Perilous. You're opening yourself up for a ton of tiny bugs. Of course, if the OP is a perfect programmer who never makes a single mistake, then sure, go ahead - it's not impossible - but I don't like the idea that you can do a mutation that works, but leaves things in a subtly inconsistent state, so the next operation (looking up an object) also works, but the one *after that* doesn't. It also means that you can't snapshot an ID to use as a reference, but can only use them *right now* (eg in a method on that object), which limits their usability. ChrisA From christopher_reimer at icloud.com Mon Jun 19 07:21:12 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Mon, 19 Jun 2017 04:21:12 -0700 Subject: Reciprocal data structures In-Reply-To: References: <59476729$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: <679D7D0A-D3D2-4DCF-86FF-90BBFCA651EF@icloud.com> > On Jun 18, 2017, at 11:02 PM, Chris Angelico wrote: > > On Mon, Jun 19, 2017 at 3:54 PM, Steven D'Aprano wrote: >>> With a list? No, I would say it's a bad idea. >> >> >> Why a bad idea? >> >> As opposed to "can't be done", or "too hard and slow". > > Maintaining a record of list indices inside an object, with the > specific proviso that: > >> If the list is changed, the list updates the indices. > A linked list with pointers to the next and previous items? If this was a homework problem, a linked list is usually implemented in C. The list keyword in Python is not the same thing. Chris R. From desrobes at gmx.com Mon Jun 19 08:22:38 2017 From: desrobes at gmx.com (ROBERTO GORINI LE REGOLE DEL DENARO) Date: Mon, 19 Jun 2017 05:22:38 -0700 (PDT) Subject: Redacted Message-ID: Redacted From mradul.k.dhakad at gmail.com Mon Jun 19 09:57:50 2017 From: mradul.k.dhakad at gmail.com (mradul dhakad) Date: Mon, 19 Jun 2017 09:57:50 -0400 Subject: Error while connecting Teradata In-Reply-To: References: Message-ID: Yes i did installed teradata odbc driver . On Mon, Jun 19, 2017 at 9:20 AM, Dennis Lee Bieber wrote: > On Sun, 18 Jun 2017 22:57:22 -0400, mradul dhakad > declaimed the following: > > > File "C:\Python27\Lib\site-packages\teradata\tdodbc.py", line 391, in > > determineDriver > > "Available drivers: {}".format(dbType, ",".join(drivers))) > > InterfaceError: ('DRIVER_NOT_FOUND', "No driver found for 'Teradata' > > > > > > >Could anyone please let me know how to resolve this Error. > > > > Uhm... Did you install the Teradata ODBC driver? > > http://downloads.teradata.com/download/connectivity/odbc-driver/windows > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > -- > https://mail.python.org/mailman/listinfo/python-list > From robin at reportlab.com Mon Jun 19 10:10:31 2017 From: robin at reportlab.com (Robin Becker) Date: Mon, 19 Jun 2017 15:10:31 +0100 Subject: Progress on the Gilectomy In-Reply-To: <87poe03k1f.fsf@nightsong.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> Message-ID: <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> On 19/06/2017 01:20, Paul Rubin wrote: ....... > the existing C API quite seriously. Reworking the C modules in the > stdlib would be a large but not impossible undertaking. The many > external C modules out there would be more of an issue. > I have always found the management of reference counts to be one of the hardest things about the C api. I'm not sure exactly how C extensions would/should interact with a GC python. There seem to be different approaches eg lua & go are both GC languages but seem different in how C/GC memory should interact. -- Robin Becker From ethan at stoneleaf.us Mon Jun 19 10:20:49 2017 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 19 Jun 2017 07:20:49 -0700 Subject: Progress on the Gilectomy In-Reply-To: <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> Message-ID: <5947DDC1.8040706@stoneleaf.us> On 06/19/2017 07:10 AM, Robin Becker wrote: > I have always found the management of reference counts to be one of the hardest things about the C api. I'm not sure > exactly how C extensions would/should interact with a GC python. There seem to be different approaches eg lua & go are > both GC languages but seem different in how C/GC memory should interact. The conversation would be easier if the proper terms were used. Reference counting is a valid garbage collecting mechanism, therefore Python is also a GC language. -- ~Ethan~ From rustompmody at gmail.com Mon Jun 19 10:21:49 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 19 Jun 2017 07:21:49 -0700 (PDT) Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> Message-ID: <0ddcc75d-65c8-4f75-adaf-f81384f876da@googlegroups.com> On Monday, June 19, 2017 at 7:40:49 PM UTC+5:30, Robin Becker wrote: > On 19/06/2017 01:20, Paul Rubin wrote: > ....... > > the existing C API quite seriously. Reworking the C modules in the > > stdlib would be a large but not impossible undertaking. The many > > external C modules out there would be more of an issue. > > > I have always found the management of reference counts to be one of the hardest > things about the C api. I'm not sure exactly how C extensions would/should > interact with a GC python. There seem to be different approaches eg lua & go are > both GC languages but seem different in how C/GC memory should interact. Worth reading for chances python missed: https://stackoverflow.com/questions/588958/what-are-the-drawbacks-of-stackless-python To be fair also this: https://stackoverflow.com/questions/377254/stackless-python-and-multicores From skip.montanaro at gmail.com Mon Jun 19 11:06:10 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 19 Jun 2017 10:06:10 -0500 Subject: Progress on the Gilectomy In-Reply-To: <5947DDC1.8040706@stoneleaf.us> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> Message-ID: On Mon, Jun 19, 2017 at 9:20 AM, Ethan Furman wrote: > Reference counting is a valid garbage collecting mechanism, therefore > Python is also a GC language. Garbage collection is usually thought of as a way to remove responsibility for tracking of live data from the user. Reference counting doesn't do that. Skip From ethan at stoneleaf.us Mon Jun 19 11:20:01 2017 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 19 Jun 2017 08:20:01 -0700 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> Message-ID: <5947EBA1.2080309@stoneleaf.us> On 06/19/2017 08:06 AM, Skip Montanaro wrote: > On Mon, Jun 19, 2017 at 9:20 AM, Ethan Furman wrote: >> Reference counting is a valid garbage collecting mechanism, therefore Python is also a GC language. > > Garbage collection is usually thought of as a way to remove responsibility for tracking of live data from the user. > Reference counting doesn't do that. Caveat: I'm not a CS major. Question: In the same way that Object Orientation is usually thought of as data hiding? Comment: Except in rare cases (e.g. messing with __del__), the Python user does not have to think about nor manage live data, so reference counting seems to meet that requirement. Programming at the C level is not working in Python, and many Python niceties simply don't exist there. -- ~Ethan~ From skip.montanaro at gmail.com Mon Jun 19 11:44:08 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 19 Jun 2017 10:44:08 -0500 Subject: Progress on the Gilectomy In-Reply-To: <5947EBA1.2080309@stoneleaf.us> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> Message-ID: On Mon, Jun 19, 2017 at 10:20 AM, Ethan Furman wrote: > Programming at the C level is not working in Python, and many Python > niceties simply don't exist there. True, but a lot of functionality available to Python programmers exists at the extension module level, whether delivered as part of the core distribution or from third-party sources. (The core CPython test suite spends a fair amount of effort on leak detection, one side effect of incorrect reference counting.) While programming in Python you don't need to worry about reference counting errors, when they slip through from the C level, they affect you. Skip From ethan at stoneleaf.us Mon Jun 19 11:48:33 2017 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 19 Jun 2017 08:48:33 -0700 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> Message-ID: <5947F251.6030005@stoneleaf.us> On 06/19/2017 08:44 AM, Skip Montanaro wrote: > On Mon, Jun 19, 2017 at 10:20 AM, Ethan Furman wrote: >> Programming at the C level is not working in Python, and many Python niceties simply don't exist there. > > True, but a lot of functionality available to Python programmers exists at the extension module level, whether delivered > as part of the core distribution or from third-party sources. (The core CPython test suite spends a fair amount of > effort on leak detection, one side effect of incorrect reference counting.) While programming in Python you don't need > to worry about reference counting errors, when they slip through from the C level, they affect you. Let me ask a different question: How much effort is required at the C level when using tracing garbage collection? -- ~Ethan~ From rgaddi at highlandtechnology.invalid Mon Jun 19 12:19:34 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 19 Jun 2017 09:19:34 -0700 Subject: Project Structure Message-ID: I'm working on a project ( https://github.com/NJDFan/register-maps http://register-maps.readthedocs.io/en/latest/ ) and running into some issues with overall project structure and documentation. The project is a code generator that reads in a set of proprietary format XML files and uses them to generate output in any number of languages (C, Python, VHDL) as well as HTML documentation. Input is an entire directory, output is an entire directory. registermap ./data/src --format=html --output=../docs/html One of the things I'd like to do is to write a boilerplate README.rst into the target directory. I also have Sphinx documentation; I'd like to incorporate all those READMEs into the HTML docs. I'd also like to have something like a registermap --format-help vhdl Which displays the appropriate README, ideally neatly formatted for the terminal and running through a pager. You know, civilized. So, right now I have a project setup which is ./ doc/ ... registermaps/ __init__.py ... resource/ vhdl/ README.rst ... html/ README.rst style.css ... README.rst setup.py ... Output classes have a .outputname, which is both the command-line output name ('html') and the directory under resource that all the stuff is in. I have a utility function in the module that allows those classes to register themselves against that name for the command-line tool. Resources are accessed through pkg_resource.resource_string internal to the program, and the READMEs get pulled into the Sphinx docs by having a doc/source/outputs.rst full of include directives like .. include:: ../../registermaps/resource/html/README.rst I still don't have any kind of standard way of pretty-printing and paging the reST to the terminal. And, more the point, this all feels VERY tangled. Like there's something that other people just know to do and I missed the memo. Anyone have any ideas on a cleaner implementation? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From mmueller at python-academy.de Mon Jun 19 12:43:31 2017 From: mmueller at python-academy.de (=?UTF-8?Q?Mike_M=c3=bcller?=) Date: Mon, 19 Jun 2017 18:43:31 +0200 Subject: PyCon.DE 2017 - Call for Proposals Message-ID: <2d822251-dbca-8170-63ae-1877bf3ab93e@python-academy.de> Call for Proposals ------------------ The Call for Proposals for the PyCon.DE 2017 is open until July 30, 2017. Please submit your proposals here: https://www.papercall.io/pyconde2017 We?re looking for proposals on every aspect of Python: programming from novice to advanced levels, applications and frameworks, or how you have been involved in introducing Python into your organization. PyCon.DE is a community conference and we are eager to hear about your experience. The conference will be held in English language only. The conferences addresses: * Data Scientists * Software Developers * System Administrators * Academic Scientists * Technology Enthusiasts We are looking for: * 30 minute presentations (incl. optional 5 minute Q&A) * 45 minute presentations (incl. optional 5 minute Q&A) PyData @ PyConDE 2017 There will be a PyData track at this year?s conference. Please submit your papers for the PyData track through the PyConDE form. About PyCon.DE 2017 ------------------- The next PyCon.DE will be held from 25-27th October 2017 at the ZKM - Center for Art and Media in Karlsruhe, Germany [1]. The conference is the sixth in this series, gathering Python professionals and enthusiasts. The conference language will English, attracting Pythonistas from all over Europe and the world. Volunteers organize and provide talks, tutorials, as well as discussions in an open-source fashion. This Karlsruhe conference has a special focus on data science. Therefore, a PyData conference is held as part of PyCon.DE. PyData [2] is an internationally renowned conference series, which is dedicated to data science, machine learning, and AI. The ZKM - Center for Art and the Media is a unique cultural institution, as it is a place that extends the original tasks of the museum. It is a home of all media and genres, a house of both the traditional arts like painting, photography, and sculpture as well as the time-based arts such as film, video, media art, music, dance, theater, and performance. Conference registration is open. Don't miss this event! More information available at: http://www.pycon.de [1] http://zkm.de [2] http://pydata.org From rosuav at gmail.com Mon Jun 19 13:04:38 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Jun 2017 03:04:38 +1000 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> Message-ID: On Tue, Jun 20, 2017 at 1:44 AM, Skip Montanaro wrote: > On Mon, Jun 19, 2017 at 10:20 AM, Ethan Furman wrote: > >> Programming at the C level is not working in Python, and many Python >> niceties simply don't exist there. > > > True, but a lot of functionality available to Python programmers exists at > the extension module level, whether delivered as part of the core > distribution or from third-party sources. (The core CPython test suite > spends a fair amount of effort on leak detection, one side effect of > incorrect reference counting.) While programming in Python you don't need > to worry about reference counting errors, when they slip through from the C > level, they affect you. High level languages mean that you don't have to write C code. Does the presence of core code and/or extension modules written in C mean that Python isn't a high level language? No. And nor does that code mean Python isn't garbage-collected. Everything has to have an implementation somewhere. Or let's look at it a different way. Instead of using a PyObject* in C code, you could write C++ code that uses a trivial wrapper class that holds the pointer, increments its refcount on construction, and decrements that refcount on destruction. That way, you can simply declare these PyObjectWrappers and let them expire. Does that mean that suddenly the refcounting isn't your responsibility, ergo it's now a garbage collector? Because the transformation is trivially easy. ChrisA From greg.ewing at canterbury.ac.nz Mon Jun 19 18:19:07 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 20 Jun 2017 10:19:07 +1200 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> Message-ID: Ethan Furman wrote: > Let me ask a different question: How much effort is required at the C > level when using tracing garbage collection? That depends on the details of the GC implementation, but often you end up swapping one form of boilerplate (maintaining ref counts) for another (such as making sure the GC system knows about all the temporary references you're using). Some, such as the Bohm collector, try to figure it all out automagically, but they rely on non-portable tricks and aren't totally reliable. -- Greg From cfkaran2 at gmail.com Mon Jun 19 20:22:22 2017 From: cfkaran2 at gmail.com (Cem Karan) Date: Mon, 19 Jun 2017 20:22:22 -0400 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> Message-ID: <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> On Jun 19, 2017, at 6:19 PM, Gregory Ewing wrote: > Ethan Furman wrote: >> Let me ask a different question: How much effort is required at the C level when using tracing garbage collection? > > That depends on the details of the GC implementation, but often > you end up swapping one form of boilerplate (maintaining ref > counts) for another (such as making sure the GC system knows > about all the temporary references you're using). > > Some, such as the Bohm collector, try to figure it all out > automagically, but they rely on non-portable tricks and aren't > totally reliable. Can you give examples of how it's not reliable? I'm currently using it in one of my projects, so if it has problems, I need to know about them. On the main topic: I think that a good tracing garbage collector would probably be a good idea. I've been having a real headache binding python to my C library via ctypes, and a large part of that problem is that I've got two different garbage collectors (python and bdwgc). I think I've got it worked out at this point, but it would have been convenient to get memory allocated from python's garbage collected heap on the C-side. Lot fewer headaches. Thanks, Cem Karan From rustompmody at gmail.com Mon Jun 19 23:52:25 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 19 Jun 2017 20:52:25 -0700 (PDT) Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> Message-ID: On Tuesday, June 20, 2017 at 5:53:00 AM UTC+5:30, Cem Karan wrote: > On Jun 19, 2017, at 6:19 PM, Gregory Ewing wrote: > > > Ethan Furman wrote: > >> Let me ask a different question: How much effort is required at the C level when using tracing garbage collection? > > > > That depends on the details of the GC implementation, but often > > you end up swapping one form of boilerplate (maintaining ref > > counts) for another (such as making sure the GC system knows > > about all the temporary references you're using). > > > > Some, such as the Bohm collector, try to figure it all out > > automagically, but they rely on non-portable tricks and aren't > > totally reliable. > > Can you give examples of how it's not reliable? I'm currently using it in one of my projects, so if it has problems, I need to know about them. Saw this this morning https://medium.com/@alexdixon/functional-programming-in-javascript-is-an-antipattern-58526819f21e May seem irrelevant to this, but if JS, FP is replaced by Python, GC it becomes more on topical From rosuav at gmail.com Tue Jun 20 00:00:36 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Jun 2017 14:00:36 +1000 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> Message-ID: On Tue, Jun 20, 2017 at 1:52 PM, Rustom Mody wrote: > Saw this this morning > https://medium.com/@alexdixon/functional-programming-in-javascript-is-an-antipattern-58526819f21e > > May seem irrelevant to this, but if JS, FP is replaced by Python, GC it becomes > more on topical https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ If super() is replaced with GC, it also becomes on-topic. I'm sure all this has some deep existential meaning about how easily blog posts can be transplanted into utterly unrelated conversations, but at the moment, it eludes me. ChrisA From no.email at nospam.invalid Tue Jun 20 00:45:09 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 19 Jun 2017 21:45:09 -0700 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> Message-ID: <87lgon2roq.fsf@nightsong.com> Lawrence D?Oliveiro writes: > Is Python 2 the Windows XP of the programming world? That's a good way to put it. It's nice to hear about Instagram but so far I don't personally know anyone who uses Python 3. Meanwhile there's still lots of new Py2 projects being started. From no.email at nospam.invalid Tue Jun 20 01:15:08 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 19 Jun 2017 22:15:08 -0700 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> Message-ID: <87h8zb2qar.fsf@nightsong.com> Chris Angelico writes: > Or let's look at it a different way. Instead of using a PyObject* in C > code, you could write C++ code that uses a trivial wrapper class that > holds the pointer, increments its refcount on construction, and > decrements that refcount on destruction. That's the C++ STL shared_ptr template. Unfortunately it has the same problem as Python refcounts, i.e. it has to use locks to maintain thread safety, which slows it down significantly. The simplest way to start experimenting with GC in Python might be to redefine the refcount macros to do nothing, connect the allocator to the Boehm GC, and stop all the threads when GC time comes. I don't know if Guile has threads at all, but I know it uses the Boehm GC and it's quite effective. From no.email at nospam.invalid Tue Jun 20 01:19:24 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 19 Jun 2017 22:19:24 -0700 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> Message-ID: <87d19z2q3n.fsf@nightsong.com> Cem Karan writes: > Can you give examples of how it's not reliable? Basically there's a chance of it leaking memory by mistaking a data word for a pointer. This is unlikely to happen by accident and usually inconsequential if it does happen, but maybe there could be malicious data that makes it happen Also, it's a non-compacting gc that has to touch all the garbage as it sweeps, not a reliability issue per se, but not great for performance especially in large, long-running systems. It's brilliant though. It's one of those things that seemingly can't possibly work, but it turns out to be quite effective. From steve at pearwood.info Tue Jun 20 01:52:31 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 20 Jun 2017 05:52:31 GMT Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> Message-ID: <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> On Mon, 19 Jun 2017 21:45:09 -0700, Paul Rubin wrote: > Lawrence D?Oliveiro writes: >> Is Python 2 the Windows XP of the programming world? > > That's a good way to put it. It's nice to hear about Instagram but so > far I don't personally know anyone who uses Python 3. Funny about that, I don't personally know anyone who uses Java or C++. You should check out Reddit's /r/python, it has a reputation for being (unreasonably, obnoxiously) pro-Python 3 even when people have a good reason for sticking with Python 1. I mean 2. > Meanwhile there's still lots of new Py2 projects being started. Unless that project is extremely small, or a throw-away ("we need it for three months, then it's obsolete"[1]), I consider starting new projects in Python 2 to be borderline professional misconduct, unless there's a genuinely good reason. (Which might include "the customer insists", or "yeah, I know it sucks, but politics".) In less than three years time, 2.7 will no longer be receiving support or bug fixes from the Python core devs. There will still be paid support from third parties available for a few years after that, but this is starting to limit your options. (E.g. are you willing to tell your clients to move their servers to RHEL so they can pay for Red Hat support for Python 2.7?) At this point, any Python developer who isn't recommending Python 3 as the default choice is doing their clients a professional disservice, in my opinion. It's still okay to use 2.7 if you have a good reason, or a bad reason for that matter, hell if you want to use 1.5 go right ahead, it's free software. But 2.7 should not be any professional Python developer's default recommendation any longer. https://pythonclock.org/ -- Steve From steve at pearwood.info Tue Jun 20 05:47:53 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 20 Jun 2017 09:47:53 GMT Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <2bca9294-3bc0-43c4-8aa5-3cdd668cb38b@googlegroups.com> <7fda44c1-a843-424b-b7cb-7f1a7de812b3@googlegroups.com> Message-ID: <5948ef49$0$1619$c3e8da3$5496439d@news.astraweb.com> On Tue, 20 Jun 2017 00:11:25 -0700, Lawrence D?Oliveiro wrote: > On Tuesday, June 20, 2017 at 6:16:05 PM UTC+12, wxjm... at gmail.com wrote: >> - Py3 on Windows just does not work. > > Whose fault is that? Pay no attention to wxjmfauth and his complaints that Python 3 does not work. He's either trolling for a reaction, or has an obsessive id?e fixe that Python 3.3+ Unicode handling is "broken" because it uses an implementation which, way back in the earliest 3.3 alpha releases when he first tested it, was slightly slower that 3.2 for some terrible benchmarks. Python 3.3 strings are optimized for memory saving: the string will use one, two or four bytes, according to the maximum needed for that particular string. This usually results in good memory savings and faster code, but if you perform an artificial benchmark of creating and destroying millions of strings as fast as you can, the overhead is a little greater than for 3.2 and it is a bit slower. In other words, if all you do is create millions of strings, destroy them, then create them again, and no more processing or work, then Python 3.3's string handling is slower than 3.2. Obviously this is a trade-off that is worth it for practical speed increases in real code due to the smaller memory usage of many Python strings. But jmfauth doesn't see it that way, and so he has invented in his own head a story that Python is "broken" because it doesn't use UTF-16 everywhere, that there's some sort of conspiracy among the Python developers to hit Europeans with higher memory use than Americans, and he complains about perfectly normal Unicode decoding or encoding exceptions as if they were segfaults. It's quite sad really, if he's trolling he is the most dedicated troll I've ever seen, continuing his efforts despite hardly any reaction from anyone. But I think he is just a crackpot. -- Steve From marko at pacujo.net Tue Jun 20 05:49:04 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 20 Jun 2017 12:49:04 +0300 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <87h8zb2qar.fsf@nightsong.com> Message-ID: <87podzxa3z.fsf@elektro.pacujo.net> Paul Rubin : > The simplest way to start experimenting with GC in Python might be to > redefine the refcount macros to do nothing, connect the allocator to > the Boehm GC, and stop all the threads when GC time comes. I don't > know if Guile has threads at all, but I know it uses the Boehm GC and > it's quite effective. Guile requires careful programming practices in the C extension code: Marko From breamoreboy at gmail.com Tue Jun 20 09:41:16 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Tue, 20 Jun 2017 06:41:16 -0700 (PDT) Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) In-Reply-To: <436e0ca8-2357-44e9-b652-b3b99ec83d1e@googlegroups.com> References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <2bca9294-3bc0-43c4-8aa5-3cdd668cb38b@googlegroups.com> <7fda44c1-a843-424b-b7cb-7f1a7de812b3@googlegroups.com> <5948ef49$0$1619$c3e8da3$5496439d@news.astraweb.com> <436e0ca8-2357-44e9-b652-b3b99ec83d1e@googlegroups.com> Message-ID: On Tuesday, June 20, 2017 at 12:18:50 PM UTC+1, wxjm... at gmail.com wrote: > Le mardi 20 juin 2017 11:48:03 UTC+2, Steven D'Aprano a ?crit?: > > Python (3) on Windows just does not work. Period. Complete drivel from the RUE. I, and many others, use Python3 on Windows on a daily basis with not problems at all. Clearly just a case of a bad workman always blames his tools. Kindest regards. Mark Lawrence. From jobmattcon at gmail.com Tue Jun 20 11:35:36 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Tue, 20 Jun 2017 08:35:36 -0700 (PDT) Subject: how to get the html content and edit with scapy and see the edited result in browser? Message-ID: <748b0cec-2159-473e-a016-308b8d44c31b@googlegroups.com> pkts = sniff(prn=lambda x:x.sprintf("{IP:%IP.src% -> %IP.dst%\n}{Raw:%Raw.load%\n}"), filter="tcp port 80") for i in range(1,len(pkts)): #if pkts[i][IP].sport == 80: i,pkts[i][TCP].payload i find pkts[10] do not have html source code (8, ) (9, ) (10, ) (11, ) dir(pkts[10][TCP]) From skip.montanaro at gmail.com Tue Jun 20 16:49:13 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 20 Jun 2017 15:49:13 -0500 Subject: cdecimal.Decimal v. decimal.Decimal Message-ID: I'd not used the Decimal classes before, but some data I was receiving from a database via pyodbc came that way. In writing some test cases, I had a hard time making things work out. I eventually figure out what was going on: >>> import decimal, cdecimal >>> decimal.Decimal('2226.48') == cdecimal.Decimal('2226.48') False (apologies for any typos, can't copy/paste) This seems odd to me. I'm currently unable to make the comparison in my Python 3 env (Conda woes), but I have both decimal and cdecimal in my Python 2.7 env. Shouldn't these beasts compare equal? Thx, Skip From ian.g.kelly at gmail.com Tue Jun 20 17:11:36 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 20 Jun 2017 15:11:36 -0600 Subject: cdecimal.Decimal v. decimal.Decimal In-Reply-To: References: Message-ID: On Tue, Jun 20, 2017 at 2:49 PM, Skip Montanaro wrote: > I'd not used the Decimal classes before, but some data I was receiving > from a database via pyodbc came that way. In writing some test cases, > I had a hard time making things work out. I eventually figure out what > was going on: > >>>> import decimal, cdecimal >>>> decimal.Decimal('2226.48') == cdecimal.Decimal('2226.48') > False > > (apologies for any typos, can't copy/paste) > > This seems odd to me. I'm currently unable to make the comparison in > my Python 3 env (Conda woes), but I have both decimal and cdecimal in > my Python 2.7 env. Shouldn't these beasts compare equal? It seems like it, although I don't think they were ever intended to be used side-by-side. cdecimal isn't part of the standard library and is supposed to be a drop-in replacement. As of CPython 3.3, the decimal module *is* cdecimal. From remmmav at gmail.com Tue Jun 20 17:47:01 2017 From: remmmav at gmail.com (remmm) Date: Tue, 20 Jun 2017 14:47:01 -0700 (PDT) Subject: Very Slow Disk Writes when Writing Large Data Blocks In-Reply-To: <5931bc86$0$740$e4fe514c@news.xs4all.nl> References: <8c383179-4396-4efd-bed2-aa8e63433226@googlegroups.com> <5931bc86$0$740$e4fe514c@news.xs4all.nl> Message-ID: <1ede1fa1-9b72-48ad-8a89-176b8b8b649e@googlegroups.com> > You'll only reach those numbers in the ideal situation. Is there just one > program doing this disk i/o, sequentially, from a single thread? The IO is sequential write of a stream of very large blocks of data onto a drive that is only say 30% full. So yes you should be able to reach 120 mbytes per second and you do on some systems. It's just that other systems including my primary system are a factor 7 to 10 slower for the same thing. > Other than that the only other thing I can think of is interference of > other programs on the system, such as malware protection or anti-virus > tooling that is trying to scan your big files at the same time. > That should be visible in Window's resource monitor tool. IT has tried totally new image -- similar results. But yes -- normally it's a corporate system with all the security bloatware. Again, we have examples of systems with all the bloatware that work... so odd. > Post it somewhere? I'll look into putting the test kit on google drive. Thanks, Rob From remmmav at gmail.com Tue Jun 20 17:52:34 2017 From: remmmav at gmail.com (remmm) Date: Tue, 20 Jun 2017 14:52:34 -0700 (PDT) Subject: Very Slow Disk Writes when Writing Large Data Blocks (Posting On Python-List Prohibited) In-Reply-To: <41ad2d73-7653-4be0-b349-beb8bb2004e5@googlegroups.com> References: <8c383179-4396-4efd-bed2-aa8e63433226@googlegroups.com> <41ad2d73-7653-4be0-b349-beb8bb2004e5@googlegroups.com> Message-ID: <2e5a4b88-dfa1-45f2-bda3-bfa37e864716@googlegroups.com> > Have you tried booting up Linux on the same hardware, and running the same tests? That would be a good way to narrow down whether the issue is hardware or software. No just other Linux systems. Hardware in question is corporate system -- so gray area as to if I can or should boot Linux ... though people do at times. Good suggestion though. Rob From jobmattcon at gmail.com Tue Jun 20 19:34:32 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Tue, 20 Jun 2017 16:34:32 -0700 (PDT) Subject: how to get the html content and edit with scapy and see the edited result in browser? In-Reply-To: <748b0cec-2159-473e-a016-308b8d44c31b@googlegroups.com> References: <748b0cec-2159-473e-a016-308b8d44c31b@googlegroups.com> Message-ID: <823383ac-cd21-41b7-9eb8-c17b7bc01ac8@googlegroups.com> then i further googled a code, but the google chrome browser and microsoft edge browser can not see the new html from the fakehttp server is there something changed to prevent edit html after intercept? originally i hope to edit https html Begin emission: .* Received 2 packets, got 1 answers, remaining 0 packets 192.168.4.52: http GET /photo/tv33.bmp HTTP/1.1 Host: 192.168.3.245 Connection: keep-alive If-None-Match: "100980-9e2bb-5526c85bb2d00" If-Modified-Since: Tue, 20 Jun 2017 23:22:17 GMT User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Accept: image/webp,image/*,*/*;q=0.8 Referer: http://192.168.3.245/ Accept-Encoding: gzip, deflate, sdch Accept-Language: en-US,en;q=0.8 Begin emission: .* Received 2 packets, got 1 answers, remaining 0 packets . Sent 1 packets. second time to run Begin emission: ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Received 3254 packets, got 0 answers, remaining 1 packets 192.168.3.245: 54302 ?hx??IDA ??s%???U????R???2???~]?N1I???????@d"A???Xy})?7AV# ???'}&?D=???)? ?K??w ?? |??? k,W???,p!?.9?@??!????&CB?#?R????Y?\gk??f ?r-??????O??=???????????c??@~Q????K??+^*a]?!RA?*6???t z? ? ?2L??1?????? ?z?Y??D?%4??PZ?!???8?? ??*? f?????K??r^??@pr?F?D?Ec????????kg ???=??0?}?q??S>???2ZwH_??????T??????? ?%[i?v?+?c?s?1 ?????????yR? ?? s 1h? ? $#$?.8?vdqbu?? ?t_? ??=3????=?? )??H??=??}???O?]'? H?_? > ??>??1?/K ?.T%#?$??S?S?????7?n?;??I=??|X{n{?/_??A??"-MD?? D^?rx{?x )N2?????O???u??+p??`???{?s?q????k9?{??x???n?7??? \Gy-?1: print GEThttp[0].load # Generate custom http file content. html1="HTTP/1.1 200 OK\x0d\x0aDate: Wed, 29 Sep 2010 20:19:05 GMT\x0d\x0aServer: Testserver\x0d\x0aConnection: Keep-Alive\x0d\x0aContent-Type: text/html; charset=UTF-8\x0d\x0aContent-Length: 291\x0d\x0a\x0d\x0aTestserver

-Welcome to test server-------------------------------

" # Generate TCP data data1=TCP(sport=80, dport=ValueOfPort, flags="PA", seq=SeqNr, ack=AckNr, options=[('MSS', 1460)]) # Construct whole network packet, send it and fetch the returning ack. ackdata1=sr1(ip/data1/html1) # Store new sequence number. SeqNr=ackdata1.ack # Generate RST-ACK packet Bye=TCP(sport=80, dport=ValueOfPort, flags="FA", seq=SeqNr, ack=AckNr, options=[('MSS', 1460)]) send(ip/Bye) from scapy.all import * import os # Interacts with a client by going through the three-way handshake. # Shuts down the connection immediately after the connection has been established. # Akaljed Dec 2010, http://www.akaljed.wordpress.com # Wait for client to connect. a=sniff(count=1,filter="tcp and host 192.168.3.245 and port 80") # some variables for later use. ValueOfPort=a[0].sport SeqNr=a[0].seq AckNr=a[0].seq+1 # Generating the IP layer: ip=IP(src="192.168.3.245", dst="192.168.100.1") # Generating TCP layer: TCP_SYNACK=TCP(sport=80, dport=ValueOfPort, flags="SA", seq=SeqNr, ack=AckNr, options=[('MSS', 1460)]) #send SYNACK to remote host AND receive ACK. ANSWER=sr1(ip/TCP_SYNACK) # Capture next TCP packets with dport 80. (contains http GET request) GEThttp = sniff(filter="tcp and port 80",count=1,prn=lambda x:x.sprintf("{IP:%IP.src%: %TCP.dport%}")) AckNr=AckNr+len(GEThttp[0].load) SeqNr=a[0].seq+1 # Print the GET request # (Sanity check: size of data should be greater than 1.) if len(GEThttp[0].load)>1: print GEThttp[0].load # Generate custom http file content. html1="HTTP/1.1 200 OK\x0d\x0aDate: Wed, 29 Sep 2010 20:19:05 GMT\x0d\x0aServer: Testserver\x0d\x0aConnection: Keep-Alive\x0d\x0aContent-Type: text/html; charset=UTF-8\x0d\x0aContent-Length: 291\x0d\x0a\x0d\x0aTestserver

-Welcome to test server-------------------------------

" # Generate TCP data data1=TCP(sport=80, dport=ValueOfPort, flags="PA", seq=SeqNr, ack=AckNr, options=[('MSS', 1460)]) # Construct whole network packet, send it and fetch the returning ack. ackdata1=sr1(ip/data1/html1) # Store new sequence number. SeqNr=ackdata1.ack # Generate RST-ACK packet Bye=TCP(sport=80, dport=ValueOfPort, flags="FA", seq=SeqNr, ack=AckNr, options=[('MSS', 1460)]) send(ip/Bye) On Tuesday, June 20, 2017 at 11:36:07 PM UTC+8, Ho Yeung Lee wrote: > pkts = sniff(prn=lambda x:x.sprintf("{IP:%IP.src% -> %IP.dst%\n}{Raw:%Raw.load%\n}"), filter="tcp port 80") > > for i in range(1,len(pkts)): > #if pkts[i][IP].sport == 80: > i,pkts[i][TCP].payload > > i find pkts[10] do not have html source code > > (8, ) > (9, ) > (10, ) > (11, ) > > dir(pkts[10][TCP]) From skip.montanaro at gmail.com Tue Jun 20 21:45:22 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 20 Jun 2017 20:45:22 -0500 Subject: cdecimal.Decimal v. decimal.Decimal In-Reply-To: References: Message-ID: Shouldn't these beasts compare equal? It seems like it, although I don't think they were ever intended to be used side-by-side. cdecimal isn't part of the standard library and is supposed to be a drop-in replacement. As of CPython 3.3, the decimal module *is* cdecimal. Got it, thanks. I have a feeling that "conda install pyodbc" snuck cdecimal in without me noticing. When I saw that the value disputed as "Decimal('2226.84')" I imported the decimal module. It took awhile to figure out I wasn't dealing with identical modules. Skip From cfkaran2 at gmail.com Tue Jun 20 21:56:56 2017 From: cfkaran2 at gmail.com (Cem Karan) Date: Tue, 20 Jun 2017 21:56:56 -0400 Subject: Progress on the Gilectomy In-Reply-To: <87d19z2q3n.fsf@nightsong.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> Message-ID: <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> On Jun 20, 2017, at 1:19 AM, Paul Rubin wrote: > Cem Karan writes: >> Can you give examples of how it's not reliable? > > Basically there's a chance of it leaking memory by mistaking a data word > for a pointer. This is unlikely to happen by accident and usually > inconsequential if it does happen, but maybe there could be malicious > data that makes it happen Got it, thank you. My processes will run for 1-2 weeks at a time, so I can handle minor memory leaks over that time without too much trouble. > Also, it's a non-compacting gc that has to touch all the garbage as it > sweeps, not a reliability issue per se, but not great for performance > especially in large, long-running systems. I'm not too sure how much of performance impact that will have. My code generates a very large number of tiny, short-lived objects at a fairly high rate of speed throughout its lifetime. At least in the last iteration of the code, garbage collection consumed less than 1% of the total runtime. Maybe this is something that needs to be done and profiled to see how well it works? > It's brilliant though. It's one of those things that seemingly can't > possibly work, but it turns out to be quite effective. Agreed! I **still** can't figure out how they managed to do it, it really does look like it shouldn't work at all! Thanks, Cem Karan From no.email at nospam.invalid Wed Jun 21 01:17:59 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 20 Jun 2017 22:17:59 -0700 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: <878tkl3omw.fsf@nightsong.com> Steven D'Aprano writes: > genuinely good reason... (Which might include "the customer insists", > or "yeah, I know it sucks, but politics".) I think the current LTS versions of Ubuntu and Debian both come with Python 2. Not sure about Centos/RHEL. Those seem like ok reasons to me. > if you want to use 1.5 go right ahead I tested all my Python 2 code for compatibility with 1.5 for quite a long time into the Python 2 series, but eventually got too addicted to stuff like using iterators pervasively. I don't think Python 2 really broke any 1.5 code though. At least the print statement still worked. I'm comfortable enough with Python that it's still what I do most of my personal stuff with, but both py2 and py3 have enough deficiencies that I see them being relegated to the numerical computation niche, where they really do have strong library support and a dedicated user base. Unfortunately, everything I know of that fixes Python's deficiencies also introduces deficiencies of its own, so at best it's a wash. From no.email at nospam.invalid Wed Jun 21 01:31:25 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 20 Jun 2017 22:31:25 -0700 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> Message-ID: <874lv93o0i.fsf@nightsong.com> Cem Karan writes: > I'm not too sure how much of performance impact that will have. My > code generates a very large number of tiny, short-lived objects at a > fairly high rate of speed throughout its lifetime. At least in the > last iteration of the code, garbage collection consumed less than 1% > of the total runtime. Maybe this is something that needs to be done > and profiled to see how well it works? If the gc uses that little runtime and your app isn't suffering from the added memory fragmentation, then it sounds like you're doing fine. > I **still** can't figure out how they managed to do it, How it works (i.e. what the implementation does) is quite simple and understandable. The amazing thing is that it doesn't leak memory catastrophically. From marko at pacujo.net Wed Jun 21 01:40:28 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 21 Jun 2017 08:40:28 +0300 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> Message-ID: <87k2456gqb.fsf@elektro.pacujo.net> Paul Rubin : > How it works (i.e. what the implementation does) is quite simple and > understandable. The amazing thing is that it doesn't leak memory > catastrophically. If I understand it correctly, the 32-bit Go language runtime implementation suffered "catastrophically" at one point. The reason was that modern programs can actually use 2GB of RAM. That being the case, there is a 50% chance for any random 4-byte combination to look like a valid pointer into the heap. Marko From rosuav at gmail.com Wed Jun 21 01:41:26 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Jun 2017 15:41:26 +1000 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) In-Reply-To: <878tkl3omw.fsf@nightsong.com> References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <878tkl3omw.fsf@nightsong.com> Message-ID: On Wed, Jun 21, 2017 at 3:17 PM, Paul Rubin wrote: > Steven D'Aprano writes: >> genuinely good reason... (Which might include "the customer insists", >> or "yeah, I know it sucks, but politics".) > > I think the current LTS versions of Ubuntu and Debian both come with > Python 2. Not sure about Centos/RHEL. Those seem like ok reasons to > me. Current LTS versions of Debian and Ubuntu come with some version of Py3, and some version of Py2.7. You can see the exact 3.x versions here: https://packages.debian.org/search?keywords=python3&searchon=names&suite=all§ion=all&exact=1 https://packages.ubuntu.com/search?keywords=python3&searchon=names&exact=1&suite=all§ion=all I don't know how to check RHEL package lists without having a license, but RHEL 4 and 5 came out before Py3 was released. So "we need to support RHEL 4" would be a legit reason to avoid Python 3 - but it's also a reason to avoid Python 2.7, as RHEL 4 ships with Python 2.3 or 2.4 or something. Also, I believe there are backports available, though I don't have proof of that. Use Python 3. ChrisA From marko at pacujo.net Wed Jun 21 02:01:32 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 21 Jun 2017 09:01:32 +0300 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <878tkl3omw.fsf@nightsong.com> Message-ID: <87tw39516r.fsf@elektro.pacujo.net> Chris Angelico : > I don't know how to check RHEL package lists without having a license, > but RHEL 4 and 5 came out before Py3 was released. So "we need to > support RHEL 4" would be a legit reason to avoid Python 3 - but it's > also a reason to avoid Python 2.7, as RHEL 4 ships with Python 2.3 or > 2.4 or something. Also, I believe there are backports available, > though I don't have proof of that. > > Use Python 3. No RHEL/CentOS release so far ships with Py3. Maybe RHEL 8 will. That's a big deal for where I work. We'd love to implement things in Python in our products but won't do it before Py3 is included on all customer platforms. RHEL 7 is among the most important Linux distributions out there, and it will be with us for years to come: . Marko From rosuav at gmail.com Wed Jun 21 02:25:55 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Jun 2017 16:25:55 +1000 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) In-Reply-To: <87tw39516r.fsf@elektro.pacujo.net> References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <878tkl3omw.fsf@nightsong.com> <87tw39516r.fsf@elektro.pacujo.net> Message-ID: On Wed, Jun 21, 2017 at 4:01 PM, Marko Rauhamaa wrote: > No RHEL/CentOS release so far ships with Py3. Maybe RHEL 8 will. > > That's a big deal for where I work. We'd love to implement things in > Python in our products but won't do it before Py3 is included on all > customer platforms. RHEL 7 is among the most important Linux > distributions out there, and it will be with us for years to come: https://access.redhat.com/support/policy/updates/errata>. By "ships with", do you mean that it's not there by default, or that you can't get it through yum? Because if it's just the former, you should be able to declare that your program depends on Python 3. RHEL 6 came out in 2010 and RHEL 7 in 2014, so I would be very surprised if there's no Python 3 available in either (the latter should have been able to pick up 3.3 or 3.4, depending on feature freeze). Unfortunately, all I can find on official sites is behind a paywall, and the info on Stack Exchange assumes that you've already seen the official info and/or can type "yum search python3" or "yum install python3" to try it. ChrisA From no.email at nospam.invalid Wed Jun 21 03:45:04 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 21 Jun 2017 00:45:04 -0700 Subject: Static typing [was Re: Python and the need for speed] References: <768f9a44-5d90-4756-8a99-821253a491f8@googlegroups.com> <58e9e519$0$1528$c3e8da3$5496439d@news.astraweb.com> <58ec9192$0$1602$c3e8da3$5496439d@news.astraweb.com> <58eceea2$0$1615$c3e8da3$5496439d@news.astraweb.com> <4277c149-e136-4d33-91d5-d011df9149f1@googlegroups.com> <58edcdd2$0$1501$c3e8da3$5496439d@news.astraweb.com> <58f2f2b1$0$1599$c3e8da3$5496439d@news.astraweb.com> <877f2jpnf7.fsf@nightsong.com> <58f49e92$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87zid1239b.fsf@nightsong.com> Gregory Ewing writes: > A JIT compiler works by observing the actual values To be pedantic, that's called a "tracing JIT". Other runtime code generation is also frequently called JIT compilation even when it's fairly stupid combining of assembly code templates, or the like. From no.email at nospam.invalid Wed Jun 21 03:49:50 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 21 Jun 2017 00:49:50 -0700 Subject: Looping [was Re: Python and the need for speed] References: <768f9a44-5d90-4756-8a99-821253a491f8@googlegroups.com> <58edcdd2$0$1501$c3e8da3$5496439d@news.astraweb.com> <54f1685d-1d2d-44b5-869e-a510569ba9d1@googlegroups.com> <1d352948-9029-4b86-b2f1-2b52608109ea@googlegroups.com> <616ebaec-f140-491c-897e-f2c4b8368d28@googlegroups.com> <58f2dc43$0$1583$c3e8da3$5496439d@news.astraweb.com> <_6JIA.1041096$Pm3.778529@fx42.am4> <87shl7mrtf.fsf@elektro.pacujo.net> <87k26jvvpn.fsf@elektro.pacujo.net> <87efwqoqtz.fsf@bsb.me.uk> <87d1caubg2.fsf@elektro.pacujo.net> <87y3uymuuq.fsf@bsb.me.uk> Message-ID: <87vanp231d.fsf@nightsong.com> Chris Angelico writes: > while True: > c = sys.stdin.read(1) > if not c: break > if c.isprintable(): text += c > elif c == "\x08": text = text[:-1] > # etc > Can you write _that_ as a do-while? I prefer to write that sort of thing with iterators: for c in iter(lambda: sys.stdin.read(1), ''): if c.isprintable(): text.append(c) elif c == '\x08': text.pop() ... From alister.ware at ntlworld.com Wed Jun 21 09:09:48 2017 From: alister.ware at ntlworld.com (alister) Date: Wed, 21 Jun 2017 13:09:48 GMT Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <2bca9294-3bc0-43c4-8aa5-3cdd668cb38b@googlegroups.com> <7fda44c1-a843-424b-b7cb-7f1a7de812b3@googlegroups.com> <5948ef49$0$1619$c3e8da3$5496439d@news.astraweb.com> <436e0ca8-2357-44e9-b652-b3b99ec83d1e@googlegroups.com> <5a519835-e24b-47e3-8531-472e893b8f76@googlegroups.com> <98729a62-30e4-46d9-a57e-8759727bdb82@googlegroups.com> Message-ID: On Wed, 21 Jun 2017 01:01:06 -0700, Lawrence D?Oliveiro wrote: > On Wednesday, June 21, 2017 at 6:59:21 PM UTC+12, wxjm... at gmail.com > wrote: >> Le mardi 20 juin 2017 15:41:27 UTC+2, bream... at gmail.com a ?crit?: >> > On Tuesday, June 20, 2017 at 12:18:50 PM UTC+1, wxjm... at gmail.com >> > wrote: >> > > Le mardi 20 juin 2017 11:48:03 UTC+2, Steven D'Aprano a ?crit?: >> > > >> > > Python (3) on Windows just does not work. Period. >> > >> > Complete drivel from the RUE. I, and many others, use Python3 on >> > Windows on a daily basis with not problems at all. Clearly just a >> > case of a bad workman always blames his tools. >> > >> > Kindest regards. >> > >> > Mark Lawrence. >> >> A very smooth example. Only the top of the iceberg. >> >> D:\junk>py32 -c "import time; print(time.tzname)" >> ('Europe de l?Ouest', 'Europe de l?Ouest (heure d??t?)') >> >> D:\junk>py33 -c "import time; print(time.tzname)" >> ('Europe de l\x92Ouest', 'Europe de l\x92Ouest (heure d\x92?t?)') > > I don?t understand where you get those strings from, or what the ?\x92? > is supposed to be. The best I can do: > > ldo at theon:~> python3.2 -c "print(('Europe de l?Ouest', 'Europe de > l?Ouest (heure d??t?)'))" > ('Europe de l?Ouest', 'Europe de l?Ouest (heure d??t?)') > ldo at theon:~> python3.3 -c "print(('Europe de l?Ouest', 'Europe de > l?Ouest (heure d??t?)'))" > ('Europe de l?Ouest', 'Europe de l?Ouest (heure d??t?)') > ldo at theon:~> python3.4 -c "print(('Europe de l?Ouest', 'Europe de > l?Ouest (heure d??t?)'))" > ('Europe de l?Ouest', 'Europe de l?Ouest (heure d??t?)') > ldo at theon:~> python3.5 -c "print(('Europe de l?Ouest', 'Europe de > l?Ouest (heure d??t?)'))" > ('Europe de l?Ouest', 'Europe de l?Ouest (heure d??t?)') > ldo at theon:~> python3.6 -c "print(('Europe de l?Ouest', 'Europe de > l?Ouest (heure d??t?)'))" > ('Europe de l?Ouest', 'Europe de l?Ouest (heure d??t?)') > > As you can see, it all works fine. Microsoft Windows trouble? don't wast time arguing with an idiot he will simply drag you down to his level then beat you with experience the only reason I have not killfiled JMX as he serves as a good guide to my understanding, if I find myself in agreement with him I assume i must be wrong/ -- Grand Master Turing once dreamed that he was a machine. When he awoke he exclaimed: "I don't know whether I am Turing dreaming that I am a machine, or a machine dreaming that I am Turing!" -- Geoffrey James, "The Tao of Programming" From torriem at gmail.com Wed Jun 21 09:43:15 2017 From: torriem at gmail.com (Michael Torrie) Date: Wed, 21 Jun 2017 07:43:15 -0600 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) In-Reply-To: References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <878tkl3omw.fsf@nightsong.com> <87tw39516r.fsf@elektro.pacujo.net> Message-ID: <75ae7f76-d1a7-726b-5c30-b38c9a7ffad4@gmail.com> On 06/21/2017 12:25 AM, Chris Angelico wrote: > By "ships with", do you mean that it's not there by default, or that > you can't get it through yum? Because if it's just the former, you > should be able to declare that your program depends on Python 3. RHEL > 6 came out in 2010 and RHEL 7 in 2014, so I would be very surprised if > there's no Python 3 available in either (the latter should have been > able to pick up 3.3 or 3.4, depending on feature freeze). Python3 is not in the official repos at all for either RHEL 6 or 7. Certainly not 4 or 5. It is, however, available in EPEL repository. EPEL is associated loosely with Red Hat, but it's by no means and officially-supported repository, and many organizations may have policies disallowing its use, or the use of any other third-party repo, for security and stability reasons. Docker might be another possibility for using and deploying Python3 apps in RHEL, although Docker itself is a third-party installation, although from a corporate point of view, since Docker is available with commercial support, it may be allowed. Also Red Hat has a repo called the "Software Collections Library" that contains up-to-date compilers and languages, including Python 3.6. However by design these packages install into specialized, self-contained environments (not in the system path or library search path) so as not to conflict in any way with system versions. As such they are a bit awkward to deal with, particularly if you want to build applications using Python 3.6 from the SCL and run them normally in RHEL. You'd need a wrapper script to set up the runtime environment before running the Python script. From mal at europython.eu Wed Jun 21 10:14:14 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Wed, 21 Jun 2017 16:14:14 +0200 Subject: EuroPython 2017: Conference app available in app stores Message-ID: We are pleased to announce our very own mobile app for the EuroPython 2017 conference: * https://ep2017.europython.eu/en/events/conference-app/ * EuroPython 2017 Conference App Engage with the conference and its attendees -------------------------------------------- The mobile app gives you access to the conference schedule (even offline), helps you in planing your conference experience (create your personal schedule) and provides a rich social engagement platform for all attendees. You can create a profile within the app (or link this to your existing social accounts), share messages and photos, and easily reach out to other fellow attendees - all from within the app. The app is available for Android, iOS and as web app for other mobile devices: https://ep2017.europython.eu/en/events/conference-app/#Install-the-EuroPython-2017-mobile-app Vital for all EuroPython attendees ---------------------------------- We will again use the conference app to keep you updated by sending updates of the schedule and inform you of important announcements via push notifications, so please consider downloading it. Many useful features -------------------- Please see our EuroPython 2017 Conference App page for more details on features and guides on how to use them: https://ep2017.europython.eu/en/events/conference-app/ Don?t forget to get your EuroPython ticket ------------------------------------------ If you want to join the EuroPython fun, be sure to get your tickets as soon as possible, since ticket sales have picked up quite a bit after we announced the schedule. https://ep2017.europython.eu/en/registration/ Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/877495457479524353 Thanks. From no.email at nospam.invalid Wed Jun 21 11:20:58 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 21 Jun 2017 08:20:58 -0700 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> Message-ID: <87r2yd1i5h.fsf@nightsong.com> Lawrence D?Oliveiro writes: > The trouble with GC is, it doesn?t know when to kick in: it just keeps > on allocating memory until it runs out. That's not how GC works, geez. Typically it would run after every N bytes of memory allocated, for N chosen to balance memory footprint with cpu overhead. From steve+python at pearwood.info Wed Jun 21 18:57:51 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 22 Jun 2017 08:57:51 +1000 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <8c033515-9aaf-4ef7-ab3c-95d382b53710@googlegroups.com> Message-ID: <594af9f0$0$1618$c3e8da3$5496439d@news.astraweb.com> On Thu, 22 Jun 2017 08:23 am, breamoreboy at gmail.com wrote: > Don't you know that Lawrence D?Oliveiro has been banned from the mailing list > as he hasn't got a clue what he's talking about, That's not why he was given a ban. Being ignorant is not a crime -- if it were, a lot more of us would be banned, including all newbies. > just like the RUE? What is your obsession with wxjmfauth? You repeatedly mention him in unrelated discussions. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Wed Jun 21 23:28:29 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 22 Jun 2017 13:28:29 +1000 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <08b5cf39-2b2a-42b8-964d-156d445e9af4@googlegroups.com> Message-ID: <594b395e$0$1606$c3e8da3$5496439d@news.astraweb.com> On Thu, 22 Jun 2017 10:30 am, Lawrence D?Oliveiro wrote: > Once again: The trouble with GC is, it doesn?t know when to kick in: it just > keeps on allocating memory until it runs out. Once again: no it doesn't. Are you aware that CPython has a GC? (Or rather, a *second* GC, apart from the reference counter.) It runs periodically to reclaim dead objects in cycles that the reference counter won't free. It runs whenever the number of allocations minus the number of deallocations exceed certain thresholds, and you can set and query the thresholds using: gc.set_threshold gc.get_threshold CPython alone disproves your assertion that GCs "keep on allocating memory until it runs out". Are you aware that there are more than one garbage collection algorithm? Apart from reference-counting GC, there are also "mark and sweep" GCs, generational GCs (like CPython's), real-time algorithms, and more. One real-time algorithm implicitly divides memory into two halves. When one half is half-full, it moves all the live objects into the other half, freeing up the first half. The Mercury programming language even has a *compile time* garbage collector that can determine when an object can be freed during compilation -- no sweeps or reference counting required. It may be that *some* (possibly toy) GC algorithms behave as you say, only running when memory is completely full. But your belief that *all* GC algorithms behave this way is simply wrong. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From no.email at nospam.invalid Thu Jun 22 00:30:16 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 21 Jun 2017 21:30:16 -0700 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <08b5cf39-2b2a-42b8-964d-156d445e9af4@googlegroups.com> Message-ID: <87mv901w6f.fsf@nightsong.com> Lawrence D?Oliveiro writes: > while ?memory footprint? depends on how much memory is actually being > retained in accessible objects. If the object won't be re-accessed but is still retained by gc, then refcounting won't free it either. > Once again: The trouble with GC is, it doesn?t know when to kick in: > it just keeps on allocating memory until it runs out. When was the last time you encountered a problem like that in practice? It's almost never an issue. "Runs out" means reached an allocation threshold that's usually much smaller than the program's memory region. And as you say, you can always manually trigger a gc if the need arises. From info at egenix.com Thu Jun 22 04:07:09 2017 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Thu, 22 Jun 2017 10:07:09 +0200 Subject: =?UTF-8?Q?ANN:_Python_Meeting_D=c3=bcsseldorf_-_28.06.2017?= Message-ID: <4dd5c647-7b4f-a571-800a-d31983979c6c@egenix.com> [This announcement is in German since it targets a local user group meeting in D?sseldorf, Germany] ________________________________________________________________________ ANK?NDIGUNG Python Meeting D?sseldorf http://pyddf.de/ Ein Treffen von Python Enthusiasten und Interessierten in ungezwungener Atmosph?re. Mittwoch, 28.06.2017, 18:00 Uhr Raum 1, 2.OG im B?rgerhaus Stadtteilzentrum Bilk D?sseldorfer Arcaden, Bachstr. 145, 40217 D?sseldorf Diese Nachricht ist auch online verf?gbar: http://www.egenix.com/company/news/Python-Meeting-Duesseldorf-2017-06-28 ________________________________________________________________________ NEUIGKEITEN * Bereits angemeldete Vortr?ge: Matthias Endler "Grumpy - Python to Go source code transcompiler and runtime" Tom Engemann "BeautifulSoup als Test framework f?r HTML" Jochen Wersd?rfer "Machine Learning: Kategorisierung von FAQs" Linus Deike "Einf?hrung in Machine Learning: Qualit?tsprognose aus Sensordaten erstellen" Andreas Bresser "Bilderkennung mit OpenCV" Philipp v.d. Bussche & Marc-Andre Lemburg "Telegram Bot als Twitter Interface: TwitterBot" Weitere Vortr?ge k?nnen gerne noch angemeldet werden: info at pyddf.de Allerdings wird vermutlich bei diesem Treffen kein Platz mehr sein, sondern erst beim n?chsten Mal im 27.09.2017. * Startzeit und Ort: Wir treffen uns um 18:00 Uhr im B?rgerhaus in den D?sseldorfer Arcaden. Das B?rgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet sich an der Seite der Tiefgarageneinfahrt der D?sseldorfer Arcaden. ?ber dem Eingang steht ein gro?es "Schwimm' in Bilk" Logo. Hinter der T?r direkt links zu den zwei Aufz?gen, dann in den 2. Stock hochfahren. Der Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt. Google Street View: http://bit.ly/11sCfiw ________________________________________________________________________ EINLEITUNG Das Python Meeting D?sseldorf ist eine regelm??ige Veranstaltung in D?sseldorf, die sich an Python Begeisterte aus der Region wendet: * http://pyddf.de/ Einen guten ?berblick ?ber die Vortr?ge bietet unser YouTube-Kanal, auf dem wir die Vortr?ge nach den Meetings ver?ffentlichen: * http://www.youtube.com/pyddf/ Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, D?sseldorf: * http://www.egenix.com/ * http://www.clark-consulting.eu/ ________________________________________________________________________ PROGRAMM Das Python Meeting D?sseldorf nutzt eine Mischung aus (Lightning) Talks und offener Diskussion. Vortr?ge k?nnen vorher angemeldet werden, oder auch spontan w?hrend des Treffens eingebracht werden. Ein Beamer mit XGA Aufl?sung steht zur Verf?gung. (Lightning) Talk Anmeldung bitte formlos per EMail an info at pyddf.de ________________________________________________________________________ KOSTENBETEILIGUNG Das Python Meeting D?sseldorf wird von Python Nutzern f?r Python Nutzer veranstaltet. Um die Kosten zumindest teilweise zu refinanzieren, bitten wir die Teilnehmer um einen Beitrag in H?he von EUR 10,00 inkl. 19% Mwst, Sch?ler und Studenten zahlen EUR 5,00 inkl. 19% Mwst. Wir m?chten alle Teilnehmer bitten, den Betrag in bar mitzubringen. ________________________________________________________________________ ANMELDUNG Da wir nur f?r ca. 20 Personen Sitzpl?tze haben, m?chten wir bitten, sich per EMail anzumelden. Damit wird keine Verpflichtung eingegangen. Es erleichtert uns allerdings die Planung. Meeting Anmeldung bitte formlos per EMail an info at pyddf.de ________________________________________________________________________ WEITERE INFORMATIONEN Weitere Informationen finden Sie auf der Webseite des Meetings: http://pyddf.de/ Mit freundlichen Gr??en, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jun 22 2017) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From kishan.sampat.cerelabs at gmail.com Thu Jun 22 07:16:16 2017 From: kishan.sampat.cerelabs at gmail.com (kishan.sampat.cerelabs at gmail.com) Date: Thu, 22 Jun 2017 04:16:16 -0700 (PDT) Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency Message-ID: I want to write a common file in which It can add the frequency by adding multiple csv file and if the same words are repeated in python then it should add the frequency in the common file can any one help me please import re import operator import string class words: def __init__(self,fh): self.fh = fh def read(self): for line in fh: yield line.split() if __name__ == "__main__": frequency = {} document_text = open('data_analysis.csv', 'r') common1_file = open("common_file1.csv", "r") text_string = document_text.read().lower() match_pattern = re.findall(r'\b[a-z]{3,15}\b', text_string) text_string_one = common1_file.read().lower() match_pattern_one = re.findall(r'\b[a-z]{3,15}\b', text_string_one) #print("match_pattern"+(str(match_pattern))) for word in match_pattern: for word1 in match_pattern_one: count = frequency.get(word,0) count1 = frequency.get(word1,0) if word1 == word: frequency[word] = count + count1 else: frequency[word] = count frequency_list = frequency.keys() text_file = open("common_file1.csv", "w") for words in frequency_list: data = (words, frequency[words]) print (data) #text_file = open("common_file1.csv", "w") #for i in data: #store_fre = (str(data)+"\n") text_file.write(str(data)+"\n") text_file.close() this is my code written by me til now but not getting satisfied results From breamoreboy at gmail.com Thu Jun 22 08:49:32 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 22 Jun 2017 05:49:32 -0700 (PDT) Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency In-Reply-To: References: Message-ID: <0d2efe80-9aaa-43fc-806a-463a32c84b65@googlegroups.com> On Thursday, June 22, 2017 at 12:16:28 PM UTC+1, kishan.samp... at gmail.com wrote: > I want to write a common file in which It can add the frequency by adding multiple csv file and if the same words are repeated in python then it should add the frequency in the common file can any one help me please > > > import re > import operator > import string > > class words: > def __init__(self,fh): > self.fh = fh > def read(self): > for line in fh: > yield line.split() > > if __name__ == "__main__": > frequency = {} > document_text = open('data_analysis.csv', 'r') > common1_file = open("common_file1.csv", "r") > > text_string = document_text.read().lower() > match_pattern = re.findall(r'\b[a-z]{3,15}\b', text_string) > > text_string_one = common1_file.read().lower() > match_pattern_one = re.findall(r'\b[a-z]{3,15}\b', text_string_one) > #print("match_pattern"+(str(match_pattern))) > for word in match_pattern: > for word1 in match_pattern_one: > count = frequency.get(word,0) > count1 = frequency.get(word1,0) > if word1 == word: > frequency[word] = count + count1 > else: > frequency[word] = count > > > frequency_list = frequency.keys() > text_file = open("common_file1.csv", "w") > for words in frequency_list: > data = (words, frequency[words]) > print (data) > #text_file = open("common_file1.csv", "w") > #for i in data: > #store_fre = (str(data)+"\n") > text_file.write(str(data)+"\n") > > > text_file.close() > > > this is my code written by me til now but not getting satisfied results A quick glance suggests you need to close common_file1.csv in before you open it in WRITE mode. You can simplify your code by using a Counter https://docs.python.org/3/library/collections.html#collections.Counter Kindest regards. Mark Lawrence From cfkaran2 at gmail.com Thu Jun 22 09:11:43 2017 From: cfkaran2 at gmail.com (CFK) Date: Thu, 22 Jun 2017 09:11:43 -0400 Subject: Progress on the Gilectomy In-Reply-To: <874lv93o0i.fsf@nightsong.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> Message-ID: On Jun 21, 2017 1:38 AM, "Paul Rubin" wrote: Cem Karan writes: > I'm not too sure how much of performance impact that will have. My > code generates a very large number of tiny, short-lived objects at a > fairly high rate of speed throughout its lifetime. At least in the > last iteration of the code, garbage collection consumed less than 1% > of the total runtime. Maybe this is something that needs to be done > and profiled to see how well it works? If the gc uses that little runtime and your app isn't suffering from the added memory fragmentation, then it sounds like you're doing fine. Yes, and this is why I suspect CPython would work well too. My usage pattern may be similar to Python usage patterns. The only way to know for sure is to try it and see what happens. > I **still** can't figure out how they managed to do it, How it works (i.e. what the implementation does) is quite simple and understandable. The amazing thing is that it doesn't leak memory catastrophically. I'll have to read through the code then, just to see what they are doing. Thanks, Cem Karan From cfkaran2 at gmail.com Thu Jun 22 09:24:34 2017 From: cfkaran2 at gmail.com (CFK) Date: Thu, 22 Jun 2017 09:24:34 -0400 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) In-Reply-To: <87mv901w6f.fsf@nightsong.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <08b5cf39-2b2a-42b8-964d-156d445e9af4@googlegroups.com> <87mv901w6f.fsf@nightsong.com> Message-ID: On Jun 22, 2017 12:38 AM, "Paul Rubin" wrote: Lawrence D?Oliveiro writes: > while ?memory footprint? depends on how much memory is actually being > retained in accessible objects. If the object won't be re-accessed but is still retained by gc, then refcounting won't free it either. > Once again: The trouble with GC is, it doesn?t know when to kick in: > it just keeps on allocating memory until it runs out. When was the last time you encountered a problem like that in practice? It's almost never an issue. "Runs out" means reached an allocation threshold that's usually much smaller than the program's memory region. And as you say, you can always manually trigger a gc if the need arises. I'm with Paul and Steve on this. I've had to do a **lot** of profiling on my simulator to get it to run at a reasonable speed. Memory usage seems to follow an exponential decay curve, hitting a strict maximum that strongly correlates with the number of live objects in a given simulation run. When I draw memory usage graphs, I see sawtooth waves to the memory usage which suggest that the garbage builds up until the GC kicks in and reaps the garbage. In short, only an exceptionally poorly written GC would exhaust memory before reaping garbage. Thanks, Cem Karan From marko at pacujo.net Thu Jun 22 09:27:39 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 22 Jun 2017 16:27:39 +0300 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> Message-ID: <8760foxid0.fsf@elektro.pacujo.net> CFK : > Yes, and this is why I suspect CPython would work well too. My usage > pattern may be similar to Python usage patterns. The only way to know for > sure is to try it and see what happens. I have a rule of thumb that your application should not need more than 10% of the available RAM. If your server has 4 GB of RAM, your application should only need 400 MB. The 90% buffer should be left for the GC to maneuver. Marko From rosuav at gmail.com Thu Jun 22 09:29:04 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 22 Jun 2017 23:29:04 +1000 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <08b5cf39-2b2a-42b8-964d-156d445e9af4@googlegroups.com> <87mv901w6f.fsf@nightsong.com> Message-ID: On Thu, Jun 22, 2017 at 11:24 PM, CFK wrote: > When > I draw memory usage graphs, I see sawtooth waves to the memory usage which > suggest that the garbage builds up until the GC kicks in and reaps the > garbage. Interesting. How do you actually measure this memory usage? Often, when a GC frees up memory, it's merely made available for subsequent allocations, rather than actually given back to the system - all it takes is one still-used object on a page and the whole page has to be retained. As such, a "create and drop" usage model would tend to result in memory usage going up for a while, but then remaining stable, as all allocations are being fulfilled from previously-released memory that's still owned by the process. ChrisA From steve+python at pearwood.info Thu Jun 22 09:33:26 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 22 Jun 2017 23:33:26 +1000 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <2bca9294-3bc0-43c4-8aa5-3cdd668cb38b@googlegroups.com> <7fda44c1-a843-424b-b7cb-7f1a7de812b3@googlegroups.com> <5948ef49$0$1619$c3e8da3$5496439d@news.astraweb.com> <7b886dfd-78c6-4580-854f-883438c46c52@googlegroups.com> Message-ID: <594bc727$0$1619$c3e8da3$5496439d@news.astraweb.com> On Wed, 21 Jun 2017 09:23 am, Lawrence D?Oliveiro wrote: > Though the Perl 6 folks claim their approach (encoding ?characters? rather > than ?code points?) is superior. Can you explain what you are referring to precisely? According to the Perl 6 docs, they do encode code points, not "characters" (which is an ill-defined concept, and besides some Unicode code points are not characters at all). http://www.unicode.org/faq/private_use.html#noncharacters For example: https://docs.perl6.org/language/unicode talks about code points. The very first section is titled "Entering Unicode Codepoints and Codepoint Sequences". Likewise there is a method "codes" which returns the number of code points in a string: https://docs.perl6.org/routine/codes On the other hand there is also a method "chars" which returns the number of "characters" (graphemes? grapheme clusters? it doesn't specify) in the string. https://docs.perl6.org/routine/chars Anyone here got Perl 6 installed and can try it out? How many "characters" does it think the string "a\uFDD5\uFDD6z" contain? - if it says 4, that's the number of code points; - if it says 2, that's the number of characters less the number of noncharacters. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Thu Jun 22 09:36:27 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 22 Jun 2017 23:36:27 +1000 Subject: Progress on the Gilectomy In-Reply-To: <8760foxid0.fsf@elektro.pacujo.net> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <8760foxid0.fsf@elektro.pacujo.net> Message-ID: On Thu, Jun 22, 2017 at 11:27 PM, Marko Rauhamaa wrote: > CFK : > >> Yes, and this is why I suspect CPython would work well too. My usage >> pattern may be similar to Python usage patterns. The only way to know for >> sure is to try it and see what happens. > > I have a rule of thumb that your application should not need more than > 10% of the available RAM. If your server has 4 GB of RAM, your > application should only need 400 MB. The 90% buffer should be left for > the GC to maneuver. *BOGGLE* I could see a justification in saying "aim for 400MB, because then unexpected spikes won't kill you", or "aim for 400MB to ensure that you can run multiple instances of the app for load balancing", or "aim for 400MB because you don't want to crowd out the database and the disk cache", but not "aim for 400MB because the garbage collector is only 10% efficient". Get yourself a better garbage collector. Employ Veolia or something. ChrisA From tomuxiong at gmx.com Thu Jun 22 09:53:04 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Thu, 22 Jun 2017 15:53:04 +0200 Subject: Best way to ensure user calls methods in correct order? Message-ID: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> Hello, I have a situation in which I want a user to call methods in a certain order and to force the re-calling of methods "down-stream" if upstream methods are called again. An example of this sort of thing would be a pipeline where calling methods again invalidates the results of methods called afterwards and you'd like to warn the user of that. Here is a very simplified example: ordered_consistency.py ----------------------------------------------- class ConsistencyError(Exception): pass class C: def __init__(self): self._a_dirty = self._b_dirty = self._c_dirty = True def a(self): self._a_dirty = self._b_dirty = self._c_dirty = True print("Calling a()...") self._a_dirty = False def b(self): if self._a_dirty: raise ConsistencyError("Re-run a() before calling b()!") self._b_dirty = self._c_dirty = True print("Calling b()...") self._b_dirty = False def c(self): if self._b_dirty: raise ConsistencyError("Re-run b() before calling c()!") self._c_dirty = True print("Calling c()...") self._c_dirty = False def d(self): if self._c_dirty: raise ConsistencyError("Re-run c() before calling d()!") print("Calling d()...") c = C() # This is fine: c.a() c.b() c.c() c.d() # This is also fine (with same class instantiation!) c.c() c.d() # This throws an error: c.b() c.d() ----------------------------------------------- Here's what you get when calling it: ----------------------------------------------- $ python3 ordered_methods.py Calling a()... Calling b()... Calling c()... Calling d()... Calling c()... Calling d()... Calling b()... Traceback (most recent call last): File "ordered_methods.py", line 43, in c.d() File "ordered_methods.py", line 29, in d raise ConsistencyError("Re-run c() before calling d()!") __main__.ConsistencyError: Re-run c() before calling d()! ----------------------------------------------- My solution seems to work, but has a large amount of repetition and errors will certainly be introduced the first time anything is changed. I have the following questions: 1) Most importantly, am I being stupid? I.e. is there some obviously better way to handle this sort of thing? 2) Is there an out of the box solution somewhere that enforces this that I've never seen before? 3) If not, can anyone here see some sort of more obvious way to do this without all the repetition in dirty bits, error messages, etc.? I presume a decorator could be used to do some sort of "order registration", but I figure I might as well ask before I re-invent a hexagonal wheel. Thanks a lot for any help! Cheers, Thomas From rosuav at gmail.com Thu Jun 22 09:57:43 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 22 Jun 2017 23:57:43 +1000 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) In-Reply-To: <594bc727$0$1619$c3e8da3$5496439d@news.astraweb.com> References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <2bca9294-3bc0-43c4-8aa5-3cdd668cb38b@googlegroups.com> <7fda44c1-a843-424b-b7cb-7f1a7de812b3@googlegroups.com> <5948ef49$0$1619$c3e8da3$5496439d@news.astraweb.com> <7b886dfd-78c6-4580-854f-883438c46c52@googlegroups.com> <594bc727$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Jun 22, 2017 at 11:33 PM, Steve D'Aprano wrote: > and besides some Unicode code points are not > characters at all). > > http://www.unicode.org/faq/private_use.html#noncharacters AIUI, "noncharacters" are like the IEEE floating point value "not-a-number". If you ask for the type of it in Python, it's "float", which is a numeric type. (It's funnier in JavaScript, where 'typeof NaN' is "number".) They're completely well-defined in terms of pretty much everything you would use a string for, the sole exception being displaying it to a human (at which point a boatload of other complexities kick in too, eg directionality (LTR/RTL), combining characters, fonts lacking certain glyphs, text wrapping, etc). So a character count should normally *include* any noncharacters in the string. But honestly, I don't know where a character count is the right choice of measurement. If you're limiting the size of user input, you probably want to count codepoints (so people don't just put five billion combining characters onto a single base), and if you're going to count combined characters, you often want to be measuring in glyphs (or maybe pixels) so it actually corresponds to the displayed text. Got any examples of where you want to count characters? And if so, do those situations govern the definition of "character"? ChrisA From __peter__ at web.de Thu Jun 22 10:31:15 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 22 Jun 2017 16:31:15 +0200 Subject: Best way to ensure user calls methods in correct order? References: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> Message-ID: Thomas Nyberg wrote: > I have a situation in which I want a user to call methods in a certain > order and to force the re-calling of methods "down-stream" if upstream > methods are called again. An example of this sort of thing would be a > pipeline where calling methods again invalidates the results of methods > called afterwards and you'd like to warn the user of that. > > Here is a very simplified example: > > ordered_consistency.py > ----------------------------------------------- > class ConsistencyError(Exception): > pass > > class C: > def __init__(self): > self._a_dirty = self._b_dirty = self._c_dirty = True > > def a(self): > self._a_dirty = self._b_dirty = self._c_dirty = True > print("Calling a()...") > self._a_dirty = False > > def b(self): > if self._a_dirty: > raise ConsistencyError("Re-run a() before calling b()!") > self._b_dirty = self._c_dirty = True > print("Calling b()...") > self._b_dirty = False > > def c(self): > if self._b_dirty: > raise ConsistencyError("Re-run b() before calling c()!") > self._c_dirty = True > print("Calling c()...") > self._c_dirty = False > > def d(self): > if self._c_dirty: > raise ConsistencyError("Re-run c() before calling d()!") > print("Calling d()...") > > c = C() > # This is fine: > c.a() > c.b() > c.c() > c.d() > # This is also fine (with same class instantiation!) > c.c() > c.d() > # This throws an error: > c.b() > c.d() > ----------------------------------------------- > > Here's what you get when calling it: > ----------------------------------------------- > $ python3 ordered_methods.py > Calling a()... > Calling b()... > Calling c()... > Calling d()... > Calling c()... > Calling d()... > Calling b()... > Traceback (most recent call last): > File "ordered_methods.py", line 43, in > c.d() > File "ordered_methods.py", line 29, in d > raise ConsistencyError("Re-run c() before calling d()!") > __main__.ConsistencyError: Re-run c() before calling d()! > ----------------------------------------------- > > My solution seems to work, but has a large amount of repetition and > errors will certainly be introduced the first time anything is changed. > I have the following questions: > > 1) Most importantly, am I being stupid? I.e. is there some obviously > better way to handle this sort of thing? > 2) Is there an out of the box solution somewhere that enforces this > that I've never seen before? > 3) If not, can anyone here see some sort of more obvious way to do this > without all the repetition in dirty bits, error messages, etc.? > > I presume a decorator could be used to do some sort of "order > registration", but I figure I might as well ask before I re-invent a > hexagonal wheel. Thanks a lot for any help! If the order is linear like it seems to be the following might work: $ cat inorder.py import itertools class ConsistencyError(Exception): pass class InOrder: def __init__(self): self.indices = itertools.count(1) def __call__(self, method): index = next(self.indices) def wrapper(self, *args, **kw): print("index", index, "state", self.state) if index - self.state > 1: raise ConsistencyError result = method(self, *args, **kw) self.state = index return result return wrapper class C: def __init__(self): self.state = 0 inorder = InOrder() @inorder def a(self): print("Calling a()...") @inorder def b(self): print("Calling b()...") @inorder def c(self): print("Calling c()...") @inorder def d(self): print("Calling d()...") $ python3 -i inorder.py >>> >>> c = C() >>> c.a() index 1 state 0 Calling a()... >>> c.b() index 2 state 1 Calling b()... >>> c.a() index 1 state 2 Calling a()... >>> c.c() index 3 state 1 Traceback (most recent call last): File "", line 1, in File "inorder.py", line 18, in wrapper raise ConsistencyError __main__.ConsistencyError >>> c.b() index 2 state 1 Calling b()... >>> c.c() index 3 state 2 Calling c()... From michael.stemper at gmail.com Thu Jun 22 10:33:15 2017 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 22 Jun 2017 09:33:15 -0500 Subject: matplotlib change? Message-ID: I have some scripts running as cronjobs that capture the status of some long-term processes and then periodically plot the data. The box where they normally run went down yesterday for some unknown reason, so I ran them manually on another box so that others on the project could continue to watch progress. I was surprised to see that the lines on the plot no longer went all of the way to its border. Investigating showed me that this is box-dependent. Samples showing the difference: good bad The names of the differing plots are based on the fact that one was done on a box with python 2.7.12 and one with python 2.7.13. (Note that the 2.7.12 box is running Ubuntu, while the 2.7.13 box is running straight Debian.) Is it likely that the difference in plots due to something that changed in matplotlib between 2.7.12 and 2.7.13? If so, is there some argument that I could specify in one of the functions to prevent this padding/margin/waste? Is there a separate function to call? If the difference isn't due to a change in matplotlib, would it be something OS-dependent? How can I track it down? Thanks for any suggestions. Appendix: Functions currently called import matplotlib.pyplot as plt plt.figure() plt.plot() plt.gca().xaxis.set_major_formatter() plt.gca().xaxis.set_major_locator() plt.legend() plt.ylabel() plt.savefig() -- Michael F. Stemper I feel more like I do now than I did when I came in. From steve+python at pearwood.info Thu Jun 22 10:40:18 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 23 Jun 2017 00:40:18 +1000 Subject: Best way to ensure user calls methods in correct order? References: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> Message-ID: <594bd6d3$0$1589$c3e8da3$5496439d@news.astraweb.com> On Thu, 22 Jun 2017 11:53 pm, Thomas Nyberg wrote: > I have a situation in which I want a user to call methods in a certain > order and to force the re-calling of methods "down-stream" if upstream > methods are called again. Don't do that. It's fragile and an anti-pattern. Your methods have too much coupling. If c() relies on b() being called first, then either b() or c() aren't good methods. They don't do enough: - calling b() alone doesn't do enough, so you have to call c() next to get the job done; - calling c() alone doesn't do enough, because it relies on b() being called first. There are a very few exceptions to this rule of thumb, such as opening connections to databases or files or similar. They are mostly enshrined from long practice, or justified by low-level APIs (that's how file systems and databases work, and it's not practical to change that). But you should try very hard to avoid creating new examples. Of course, I'm talking about your *public* methods. Private methods can be a bit more restrictive, since it's only *you* who suffers if you do it wrong. Ideally, your methods should be written in a functional style with as little shared state as possible. (Shared state introduces coupling between components, and excessive coupling is *the* ultimate evil in programming.) With no shared state, your methods are trivial: def a(self, arg): return something def b(self, arg): # relies on a x = self.a(arg) return process(x) def c(self, arg): # relies on b x = self.b(arg) return process(x) With shared state, it becomes more horrible, but at least your users are shared the pain. (*You* on the other hand, are not -- that's your punishment for writing in a Java-like style with lots of shared state.) def a(self): self.data = something self.step = 1 def b(self): # relies on a assert self.step >= 0 if self.step == 0: self.a() if self.step == 1: self.data = process(self.data) self.step = 2 else: assert self.step > 1 # do nothing? # or raise? def c(self): # relies on b assert self.step >= 0 if self.step < 2: self.b() if self.step == 2: self.data = process(self.data) self.step = 3 else: assert self.step > 1 # do nothing? # or raise? This is messy enough if there is only a single chain of correct calls: # you must call a, b, c, d in that order If you can call the methods in lots of different ways: # you can call a, b, c, d; # or a, b, d; # or a, d, e, f # or g, d, e # ... it becomes horrible. In that case, I'd say just document what the methods do and let the user deal with the mess. > ????????1) Most importantly, am I being stupid? I.e. is there some obviously > better way to handle this sort of thing? I wouldn't say "stupid" as such, but since you used the word, yes :-) > ????????2) Is there an out of the box solution somewhere that enforces this > that I've never seen before? > ????????3) If not, can anyone here see some sort of more obvious way to do > this without all the repetition in dirty bits, error messages, etc.? If you want the user to call methods a, b, c, d in that order, provide then with a single method "run" that calls a, b, c, d in that order, and tell them to call "run". -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From breamoreboy at gmail.com Thu Jun 22 10:50:24 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 22 Jun 2017 07:50:24 -0700 (PDT) Subject: matplotlib change? In-Reply-To: References: Message-ID: <5f81ee9e-8ac1-480f-8110-ae129f3837fd@googlegroups.com> On Thursday, June 22, 2017 at 3:33:36 PM UTC+1, Michael F. Stemper wrote: > I have some scripts running as cronjobs that capture the status > of some long-term processes and then periodically plot the data. > The box where they normally run went down yesterday for some > unknown reason, so I ran them manually on another box so that > others on the project could continue to watch progress. > > I was surprised to see that the lines on the plot no longer went > all of the way to its border. Investigating showed me that this > is box-dependent. > > Samples showing the difference: > > good > > > bad > > > The names of the differing plots are based on the fact that one > was done on a box with python 2.7.12 and one with python 2.7.13. > (Note that the 2.7.12 box is running Ubuntu, while the 2.7.13 box > is running straight Debian.) > > Is it likely that the difference in plots due to something that > changed in matplotlib between 2.7.12 and 2.7.13? If so, is there > some argument that I could specify in one of the functions to > prevent this padding/margin/waste? Is there a separate function > to call? > > If the difference isn't due to a change in matplotlib, would it be > something OS-dependent? How can I track it down? > > Thanks for any suggestions. > > Appendix: Functions currently called > > import matplotlib.pyplot as plt > plt.figure() > plt.plot() > plt.gca().xaxis.set_major_formatter() > plt.gca().xaxis.set_major_locator() > plt.legend() > plt.ylabel() > plt.savefig() > > -- > Michael F. Stemper > I feel more like I do now than I did when I came in. Id check to see which matplotlib versions you have rather than the Python version. Either:- C:\python Python 3.6.2rc1 (heads/3.6:268e1fb, Jun 17 2017, 19:01:44) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import matplotlib >>> matplotlib.__version__ '2.0.0' >>> or:- C:\python -c "import matplotlib;print(matplotlib.__version__)" 2.0.0 Kindest regards. Mark Lawrence. From steve+python at pearwood.info Thu Jun 22 10:50:36 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 23 Jun 2017 00:50:36 +1000 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) References: <8760fsk4sb.fsf@nightsong.com> <0b495831-737a-4540-abcb-b5d19a32f132@googlegroups.com> <87lgon2roq.fsf@nightsong.com> <5948b81f$0$1619$c3e8da3$5496439d@news.astraweb.com> <2bca9294-3bc0-43c4-8aa5-3cdd668cb38b@googlegroups.com> <7fda44c1-a843-424b-b7cb-7f1a7de812b3@googlegroups.com> <5948ef49$0$1619$c3e8da3$5496439d@news.astraweb.com> <7b886dfd-78c6-4580-854f-883438c46c52@googlegroups.com> <594bc727$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: <594bd93e$0$1604$c3e8da3$5496439d@news.astraweb.com> On Thu, 22 Jun 2017 11:57 pm, Chris Angelico wrote: > On Thu, Jun 22, 2017 at 11:33 PM, Steve D'Aprano > wrote: >> and besides some Unicode code points are not >> characters at all). >> >> http://www.unicode.org/faq/private_use.html#noncharacters > > AIUI, "noncharacters" are like the IEEE floating point value > "not-a-number". That's... kinda fair. Although, the Unicode Consortium thinks of them as more like private use characters, only even more private, and not characters :-) (If you ask me, I think the noncharacters exist because "it seemed like a good idea at the time" -- the use-case for them seems particularly ill-defined. I suspect that if we were to redo Unicode from scratch, they wouldn't be included.) > So a character count should normally *include* any noncharacters in the > string. That depends on what you mean by *character*. If you mean "code point", then I agree it should be counted. If you mean "a letter, a digit, an ideograph, emoji, ... " then probably not. (Depends what's in the ellipsis :-) If you mean a grapheme, then certainly not, because the 66 Unicode noncharacters don't belong to any human language. If you mean "a grapheme cluster, or a code point for things which aren't characters from human languages" then I guess they should be counted, as will control characters, formatting marks, surrogate code points, and anything else which doesn't represent a natural language character. What is a natural language character? Is IJ one or two characters? Depends on whether you're Dutch or not ;-) This is why the Unicode standard tries not to talk in terms of "characters". They're not well-defined. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tomuxiong at gmx.com Thu Jun 22 11:31:46 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Thu, 22 Jun 2017 17:31:46 +0200 Subject: Best way to ensure user calls methods in correct order? In-Reply-To: <594bd6d3$0$1589$c3e8da3$5496439d@news.astraweb.com> References: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> <594bd6d3$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: Thanks for the response! I see the reasoning, but I can't entirely square it with what I'm thinking. Probably it's either because I was hiding some of my intention (sorry if I wasted your time due to lack of details) or that I'm naive and regardless of intention doing something silly. On 06/22/2017 04:40 PM, Steve D'Aprano wrote: > On Thu, 22 Jun 2017 11:53 pm, Thomas Nyberg wrote: > > Don't do that. It's fragile and an anti-pattern. Your methods have too much > coupling. If c() relies on b() being called first, then either b() or c() > aren't good methods. They don't do enough: > > - calling b() alone doesn't do enough, so you have to call c() next to get the > job done; > > - calling c() alone doesn't do enough, because it relies on b() being called > first. > > > There are a very few exceptions to this rule of thumb, such as opening > connections to databases or files or similar. They are mostly enshrined from > long practice, or justified by low-level APIs (that's how file systems and > databases work, and it's not practical to change that). But you should try very > hard to avoid creating new examples. > > Of course, I'm talking about your *public* methods. Private methods can be a bit > more restrictive, since it's only *you* who suffers if you do it wrong. > > Ideally, your methods should be written in a functional style with as little > shared state as possible. (Shared state introduces coupling between components, > and excessive coupling is *the* ultimate evil in programming.) > This makes perfect sense in general. An important detail I left out earlier was what exactly I meant by "users" of my code. Really all I meant was myself calling this code as a library and avoiding making mistakes. Currently what I'm doing is kind of a data storage, data processing/conversion, data generation pipeline. This different variables at the different steps in the pipeline are specified in a script. The functions in this script have _no coupling at all_ for all the reasons you stated. In fact, the only reason I'm introducing a class is because I'm trying to force their ordering in the way I described. The ultimate goal here is to instead put a basic http server wrapper around the whole process. The reason I want to do this is to allow others (internal to the company) to make changes in the process interactively at different steps in the process (the steps are "natural" for the problem at hand) without editing the python scripts explicitly. Of course I could force them to execute everything in one go, but due to the time required for the different steps, it's nicer to allow for some eye-balling of results (and possible changes and re-running) before continuing. Before this the main way of enforcing this was all done by the few of use editing the scripts, but now I'd like to make it an error for corrupted data to propagate down through the pipeline. Having a run_all() method or something similar will definitely exist (usually once you figure out the "right" parameters you don't change them much on re-runs), but having things broken apart is still very nice. Would you still say this is a bad way of going about it? Thanks for the feedback. It's very helpful. Cheers, Thomas From mal at europython.eu Thu Jun 22 11:34:52 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Thu, 22 Jun 2017 17:34:52 +0200 Subject: EuroPython 2017: Call for on-site volunteers Message-ID: <25af4a6f-c8fa-a560-2902-e4e07a230a02@europython.eu> Would you like to be more than a participant and contribute to make this 2017 edition of EuroPython a smooth success? Help us! * EuroPython 2017 Volunteers * https://ep2017.europython.eu/en/registration/volunteers/ We have a few tasks that are open for attendees who would like to volunteer: fancy helping at the registration desk? Willing to chair a session? Find out how you can contribute and which task you can commit to. What kind of qualifications do you need ? ----------------------------------------- English is a requirement. More languages are an advantage. Check our webpage or write us for any further information. The conference ticket is a requirement. We cannot give you a free ticket, but we would like to thank you with one of our volunteer perks. How do you sign up ? -------------------- You can sign up for activities on our EuroPython 2017 Volunteer App: https://ep2017.europython.eu/volunteer-app We really appreciate your help ! Don?t forget to get your EuroPython ticket ------------------------------------------ If you want to join the EuroPython fun, be sure to get your tickets as soon as possible, since ticket sales have picked up quite a bit after we announced the schedule. https://ep2017.europython.eu/en/registration/ Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/877911423791362052 Thanks. From tomuxiong at gmx.com Thu Jun 22 11:35:36 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Thu, 22 Jun 2017 17:35:36 +0200 Subject: Best way to ensure user calls methods in correct order? In-Reply-To: References: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> Message-ID: <2fc6dbba-f182-4cfb-19e4-d757e2918a3f@gmx.com> [I accidentally sent this only to Peter, but wanted instead to send it to the list for anyone else who happens upon it...now sending to the list...] On 06/22/2017 04:31 PM, Peter Otten wrote: > If the order is linear like it seems to be the following might work: Thanks a lot! This removes the duplication that was bothering me. To get the error messages I wanted, I changed the code minorly as follows: ------------------------------------------ import itertools class ConsistencyError(Exception): pass class InOrder: def __init__(self): self.indices = itertools.count(1) self.prior_method_name = "" def __call__(self, method): index = next(self.indices) prior_method_name = self.prior_method_name method_name = method.__name__ self.prior_method_name = method_name def wrapper(this, *args, **kw): if index - this.state > 1: msg = "call {}() before calling {}()".format( prior_method_name, method_name) raise ConsistencyError(msg) result = method(this, *args, **kw) this.state = index return result return wrapper class C: def __init__(self): self.state = 0 inorder = InOrder() @inorder def a(self): print("Calling a()...") @inorder def b(self): print("Calling b()...") @inorder def c(self): print("Calling c()...") @inorder def d(self): print("Calling d()...") ------------------------------------------ Now of course I'll have to consider Steve's response and wonder _why_ exactly I'm doing this... Cheers, Thomas From rosuav at gmail.com Thu Jun 22 11:36:50 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Jun 2017 01:36:50 +1000 Subject: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited) In-Reply-To: References: <7fda44c1-a843-424b-b7cb-7f1a7de812b3@googlegroups.com> <5948ef49$0$1619$c3e8da3$5496439d@news.astraweb.com> <7b886dfd-78c6-4580-854f-883438c46c52@googlegroups.com> <594bc727$0$1619$c3e8da3$5496439d@news.astraweb.com> <594bd93e$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Jun 23, 2017 at 1:14 AM, Dennis Lee Bieber wrote: > On Fri, 23 Jun 2017 00:50:36 +1000, Steve D'Aprano > declaimed the following: > >> >>Although, the Unicode Consortium thinks of them as more like private use >>characters, only even more private, and not characters :-) >> >>(If you ask me, I think the noncharacters exist because "it seemed like a good >>idea at the time" -- the use-case for them seems particularly ill-defined. I >>suspect that if we were to redo Unicode from scratch, they wouldn't be >>included.) >> > Almost sounds like the Unicode equivalent of the private LAN IP > addresses (192.168.x.x, 10.x.x.x, whatever the "class B" address is) > > IE: available for your use but should never get out into the wild. No, because there are "private use areas" for that. :) ChrisA From marko at pacujo.net Thu Jun 22 11:48:48 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 22 Jun 2017 18:48:48 +0300 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <8760foxid0.fsf@elektro.pacujo.net> Message-ID: <87wp84vx9b.fsf@elektro.pacujo.net> Chris Angelico : > not "aim for 400MB because the garbage collector is only 10% > efficient". Get yourself a better garbage collector. Employ Veolia or > something. It's about giving GC room (space- and timewise) to operate. Also, you don't want your memory consumption to hit the RAM ceiling even for a moment. Marko From marko at pacujo.net Thu Jun 22 11:54:47 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 22 Jun 2017 18:54:47 +0300 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <8760foxid0.fsf@elektro.pacujo.net> <87wp84vx9b.fsf@elektro.pacujo.net> Message-ID: <87o9tgvwzc.fsf@elektro.pacujo.net> Marko Rauhamaa : > Chris Angelico : > >> not "aim for 400MB because the garbage collector is only 10% >> efficient". Get yourself a better garbage collector. Employ Veolia or >> something. > > It's about giving GC room (space- and timewise) to operate. Also, you > don't want your memory consumption to hit the RAM ceiling even for a > moment. And, BTW, my rule of thumb came from experiences with the Hotspot JRE. Marko From rosuav at gmail.com Thu Jun 22 12:03:20 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Jun 2017 02:03:20 +1000 Subject: Progress on the Gilectomy In-Reply-To: <87wp84vx9b.fsf@elektro.pacujo.net> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <8760foxid0.fsf@elektro.pacujo.net> <87wp84vx9b.fsf@elektro.pacujo.net> Message-ID: On Fri, Jun 23, 2017 at 1:48 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> not "aim for 400MB because the garbage collector is only 10% >> efficient". Get yourself a better garbage collector. Employ Veolia or >> something. > > It's about giving GC room (space- and timewise) to operate. Also, you > don't want your memory consumption to hit the RAM ceiling even for a > moment. Again, if you'd said to *leave 10% room*, I would be inclined to believe you (eg to use no more than 3.5ish gig when you have four available), but not to leave 90% room. ChrisA From michael.stemper at gmail.com Thu Jun 22 13:14:10 2017 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 22 Jun 2017 12:14:10 -0500 Subject: matplotlib change? In-Reply-To: <5f81ee9e-8ac1-480f-8110-ae129f3837fd@googlegroups.com> References: <5f81ee9e-8ac1-480f-8110-ae129f3837fd@googlegroups.com> Message-ID: On 2017-06-22 09:50, breamoreboy at gmail.com wrote: > On Thursday, June 22, 2017 at 3:33:36 PM UTC+1, Michael F. Stemper wrote: >> I have some scripts running as cronjobs that capture the status >> of some long-term processes and then periodically plot the data. >> The box where they normally run went down yesterday for some >> unknown reason, so I ran them manually on another box so that >> others on the project could continue to watch progress. >> >> I was surprised to see that the lines on the plot no longer went >> all of the way to its border. Investigating showed me that this >> is box-dependent. >> Is it likely that the difference in plots due to something that >> changed in matplotlib between 2.7.12 and 2.7.13? If so, is there >> some argument that I could specify in one of the functions to >> prevent this padding/margin/waste? Is there a separate function >> to call? >> >> If the difference isn't due to a change in matplotlib, would it be >> something OS-dependent? How can I track it down? > Id check to see which matplotlib versions you have rather than the Python version. Either:- Okay, that was easy enough: mstemper2 at greenbay$ python Python 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import matplotlib >>> matplotlib.__version__ '1.5.1' >>> mstemper2 at vv322f$ python Python 2.7.13 (default, Jan 19 2017, 14:48:08) [GCC 6.3.0 20170118] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import matplotlib >>> matplotlib.__version__ '2.0.0' >>> What's my next step? -- Michael F. Stemper Deuteronomy 24:17 From michael.stemper at gmail.com Thu Jun 22 13:16:50 2017 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 22 Jun 2017 12:16:50 -0500 Subject: matplotlib change? In-Reply-To: References: Message-ID: On 2017-06-22 10:54, Dennis Lee Bieber wrote: > On Thu, 22 Jun 2017 09:33:15 -0500, "Michael F. Stemper" > declaimed the following: >> If the difference isn't due to a change in matplotlib, would it be >> something OS-dependent? How can I track it down? >> > What renderer is being used? Tk, wx, etc. -- and are those libraries > the same on the systems? I would have thought that matplotlib or pyplot would have been doing the rendering. Since that's apparently not the case, how would I investigate? > Are you only saving to files, or rendering on screen? The script runs unattended on a crontab, so it saves to a file. -- Michael F. Stemper Deuteronomy 24:17 From bill.janssen at gmail.com Thu Jun 22 13:56:26 2017 From: bill.janssen at gmail.com (bill.janssen at gmail.com) Date: Thu, 22 Jun 2017 10:56:26 -0700 (PDT) Subject: matplotlib change? In-Reply-To: References: <5f81ee9e-8ac1-480f-8110-ae129f3837fd@googlegroups.com> Message-ID: <44af14a2-d4c0-4a49-a0d4-652375e36070@googlegroups.com> On Thursday, June 22, 2017 at 10:14:21 AM UTC-7, Michael F. Stemper wrote: > On 2017-06-22 09:50, breamoreboy at gmail.com wrote: > > On Thursday, June 22, 2017 at 3:33:36 PM UTC+1, Michael F. Stemper wrote: > >> I have some scripts running as cronjobs that capture the status > >> of some long-term processes and then periodically plot the data. > >> The box where they normally run went down yesterday for some > >> unknown reason, so I ran them manually on another box so that > >> others on the project could continue to watch progress. > >> > >> I was surprised to see that the lines on the plot no longer went > >> all of the way to its border. Investigating showed me that this > >> is box-dependent. > > >> Is it likely that the difference in plots due to something that > >> changed in matplotlib between 2.7.12 and 2.7.13? If so, is there > >> some argument that I could specify in one of the functions to > >> prevent this padding/margin/waste? Is there a separate function > >> to call? > >> > >> If the difference isn't due to a change in matplotlib, would it be > >> something OS-dependent? How can I track it down? > > > Id check to see which matplotlib versions you have rather than the Python version. Either:- > > Okay, that was easy enough: > > mstemper2 at greenbay$ python > Python 2.7.12 (default, Nov 19 2016, 06:48:10) > [GCC 5.4.0 20160609] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import matplotlib > >>> matplotlib.__version__ > '1.5.1' > >>> > > mstemper2 at vv322f$ python > Python 2.7.13 (default, Jan 19 2017, 14:48:08) > [GCC 6.3.0 20170118] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import matplotlib > >>> matplotlib.__version__ > '2.0.0' > >>> > > What's my next step? > > -- > Michael F. Stemper > Deuteronomy 24:17 There were a number of changes to line-drawing in the bump from 1 to 2 -- I found that some of them were not listed in the change notes. It's worth reviewing https://matplotlib.org/users/dflt_style_changes.html#plotting-functions. Bill From michael.stemper at gmail.com Thu Jun 22 14:24:28 2017 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 22 Jun 2017 13:24:28 -0500 Subject: matplotlib change? In-Reply-To: <44af14a2-d4c0-4a49-a0d4-652375e36070@googlegroups.com> References: <5f81ee9e-8ac1-480f-8110-ae129f3837fd@googlegroups.com> <44af14a2-d4c0-4a49-a0d4-652375e36070@googlegroups.com> Message-ID: On 2017-06-22 12:56, bill.janssen at gmail.com wrote: > On Thursday, June 22, 2017 at 10:14:21 AM UTC-7, Michael F. Stemper wrote: >> On 2017-06-22 09:50, breamoreboy at gmail.com wrote: >>> On Thursday, June 22, 2017 at 3:33:36 PM UTC+1, Michael F. Stemper wrote: > >>>> Is it likely that the difference in plots due to something that >>>> changed in matplotlib between 2.7.12 and 2.7.13? If so, is there >>>> some argument that I could specify in one of the functions to >>>> prevent this padding/margin/waste? Is there a separate function >>>> to call? >>>> >>>> If the difference isn't due to a change in matplotlib, would it be >>>> something OS-dependent? How can I track it down? >> >>> Id check to see which matplotlib versions you have rather than the Python version. Either:- >> >> Okay, that was easy enough: >> >>> import matplotlib >> >>> matplotlib.__version__ >> '2.0.0' >> >>> >> >> What's my next step? > There were a number of changes to line-drawing in the bump from 1 to 2 -- I found that some of them were not listed in the change notes. It's worth reviewing https://matplotlib.org/users/dflt_style_changes.html#plotting-functions. That nailed it! This section: addressed my exact problem. (Unfortunately, the parameters are not back-compatible, but at least I'll be ready when they upgrade the box where I usually work.) Thanks for your help. BTW, your name is familiar. Were you ever at PARC? -- Michael F. Stemper This post contains greater than 95% post-consumer bytes by weight. From michael.stemper at gmail.com Thu Jun 22 14:55:59 2017 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 22 Jun 2017 13:55:59 -0500 Subject: matplotlib change? In-Reply-To: References: Message-ID: On 2017-06-22 12:16, Michael F. Stemper wrote: > On 2017-06-22 10:54, Dennis Lee Bieber wrote: >> On Thu, 22 Jun 2017 09:33:15 -0500, "Michael F. Stemper" >> declaimed the following: > >>> If the difference isn't due to a change in matplotlib, would it be >>> something OS-dependent? How can I track it down? >>> >> What renderer is being used? Tk, wx, etc. -- and are those libraries >> the same on the systems? > > I would have thought that matplotlib or pyplot would have been doing the > rendering. Since that's apparently not the case, how would I > investigate? Never mind. Somebody found the solution cross-thread. Thanks for your time. -- Michael F. Stemper A preposition is something you should never end a sentence with. From cfkaran2 at gmail.com Thu Jun 22 15:22:31 2017 From: cfkaran2 at gmail.com (CFK) Date: Thu, 22 Jun 2017 15:22:31 -0400 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <08b5cf39-2b2a-42b8-964d-156d445e9af4@googlegroups.com> <87mv901w6f.fsf@nightsong.com> Message-ID: On Jun 22, 2017 9:32 AM, "Chris Angelico" wrote: On Thu, Jun 22, 2017 at 11:24 PM, CFK wrote: > When > I draw memory usage graphs, I see sawtooth waves to the memory usage which > suggest that the garbage builds up until the GC kicks in and reaps the > garbage. Interesting. How do you actually measure this memory usage? Often, when a GC frees up memory, it's merely made available for subsequent allocations, rather than actually given back to the system - all it takes is one still-used object on a page and the whole page has to be retained. As such, a "create and drop" usage model would tend to result in memory usage going up for a while, but then remaining stable, as all allocations are being fulfilled from previously-released memory that's still owned by the process. I'm measuring it using a bit of a hack; I use psutil.Popen ( https://pypi.python.org/pypi/psutil) to open a simulation as a child process, and in a tight loop gather the size of the resident set and the number of virtual pages currently in use of the child. The sawtooths are about 10% (and decreasing) of the size of the overall memory usage, and are probably due to different stages of the simulation doing different things. That is an educated guess though, I don't have strong evidence to back it up. And, yes, what you describe is pretty close to what I'm seeing. The longer the simulation has been running, the smoother the memory usage gets. Thanks, Cem Karan From ned at nedbatchelder.com Thu Jun 22 15:27:11 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 22 Jun 2017 12:27:11 -0700 (PDT) Subject: Progress on the Gilectomy In-Reply-To: <3861efa9-ceaa-49f7-95b4-b38a0e14c9bc@googlegroups.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <8c033515-9aaf-4ef7-ab3c-95d382b53710@googlegroups.com> <594af9f0$0$1618$c3e8da3$5496439d@news.astraweb.com> <3861efa9-ceaa-49f7-95b4-b38a0e14c9bc@googlegroups.com> Message-ID: On Thursday, June 22, 2017 at 11:07:36 AM UTC-4, bream... at gmail.com wrote: > On Wednesday, June 21, 2017 at 11:58:03 PM UTC+1, Steve D'Aprano wrote: > > On Thu, 22 Jun 2017 08:23 am, breamoreboy wrote: > > > > > Don't you know that Lawrence D?Oliveiro has been banned from the mailing list > > > as he hasn't got a clue what he's talking about, > > > > That's not why he was given a ban. Being ignorant is not a crime -- if it were, > > a lot more of us would be banned, including all newbies. > > > > > just like the RUE? > > > > What is your obsession with wxjmfauth? You repeatedly mention him in unrelated > > discussions. > > > > 11 comments on the thread "Instagram: 40% Py3 to 99% Py3 in 10 months" showing that he knows as much about Unicode as LDO knows about garabge collection. You've been asked to stop making personal attacks before. Please stop. --Ned. From jussi.piitulainen at helsinki.fi Thu Jun 22 15:46:28 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 22 Jun 2017 22:46:28 +0300 Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency References: <1qtnkcp6k42qg2peg4174shsv0c2a4bg8a@4ax.com> Message-ID: Dennis Lee Bieber writes: > # lowerecase all, open hyphenated and / separated words, parens, > # etc. > ln = ln.lower().replace("/", " ").replace("-", " ").replace(".", " ") > ln = ln.replace("\\", " ").replace("[", " ").replace("]", " ") > ln = ln.replace("{", " ").replace("}", " ") > wds = ln.replace("(", " ").replace(")", " ").replace("\t", " ").split() A pair of methods, str.maketrans to make a translation table and then .translate on every string, allows to do all that in one step: spacy = r'\/-.[]{}()' tr = str.maketrans(dict.fromkeys(spacy, ' ')) ... ln = ln.translate(tr) But those seem to be only in Python 3. > # for each word in the line > for wd in wds: > # strip off leading/trailing punctuation > wd = wd.strip("\\|'\";'[]{},<>?~!@#$%^&*_+= ") You have already replaced several of those characters with spaces. > # do we still have a word? Skip any with still embedded > # punctuation > if wd and wd.isalpha(): > # attempt to update the count for this word But for quick and dirty work I might use a very simple regex, probably literally this regex: wordy = re.compile(r'\w+') ... for wd in wordy.findall(ln): # or .finditer, but I think it's newer ... However, if the OP really is getting their input from a CSV file, they shouldn't need methods like these. Because surely it's then already an unambiguous list of words, to be read in with the csv module? Or else it's not yet CSV at all after all? I think they need to sit down with someone who can walk them through the whole exercise. From rosuav at gmail.com Thu Jun 22 15:59:30 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Jun 2017 05:59:30 +1000 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <08b5cf39-2b2a-42b8-964d-156d445e9af4@googlegroups.com> <87mv901w6f.fsf@nightsong.com> Message-ID: On Fri, Jun 23, 2017 at 5:22 AM, CFK wrote: > On Jun 22, 2017 9:32 AM, "Chris Angelico" wrote: > > On Thu, Jun 22, 2017 at 11:24 PM, CFK wrote: >> When >> I draw memory usage graphs, I see sawtooth waves to the memory usage which >> suggest that the garbage builds up until the GC kicks in and reaps the >> garbage. > > Interesting. How do you actually measure this memory usage? Often, > when a GC frees up memory, it's merely made available for subsequent > allocations, rather than actually given back to the system - all it > takes is one still-used object on a page and the whole page has to be > retained. > > As such, a "create and drop" usage model would tend to result in > memory usage going up for a while, but then remaining stable, as all > allocations are being fulfilled from previously-released memory that's > still owned by the process. > > > I'm measuring it using a bit of a hack; I use psutil.Popen > (https://pypi.python.org/pypi/psutil) to open a simulation as a child > process, and in a tight loop gather the size of the resident set and the > number of virtual pages currently in use of the child. The sawtooths are > about 10% (and decreasing) of the size of the overall memory usage, and are > probably due to different stages of the simulation doing different things. > That is an educated guess though, I don't have strong evidence to back it > up. > > And, yes, what you describe is pretty close to what I'm seeing. The longer > the simulation has been running, the smoother the memory usage gets. Ah, I think I understand. So the code would be something like this: Phase one: Create a bunch of objects Do a bunch of simulation Destroy a bunch of objects Simulate more Destroy all the objects used in this phase, other than the result Phase two: Like phase one In that case, yes, it's entirely possible that the end of a phase could signal a complete cleanup of intermediate state, with the consequent release of memory to the system. (Or, more likely, a near-complete cleanup, with release of MOST of memory.) Very cool bit of analysis you've done there. ChrisA From cfkaran2 at gmail.com Thu Jun 22 16:48:57 2017 From: cfkaran2 at gmail.com (CFK) Date: Thu, 22 Jun 2017 16:48:57 -0400 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <08b5cf39-2b2a-42b8-964d-156d445e9af4@googlegroups.com> <87mv901w6f.fsf@nightsong.com> Message-ID: On Jun 22, 2017 4:03 PM, "Chris Angelico" wrote: On Fri, Jun 23, 2017 at 5:22 AM, CFK wrote: > On Jun 22, 2017 9:32 AM, "Chris Angelico" wrote: > > On Thu, Jun 22, 2017 at 11:24 PM, CFK wrote: >> When >> I draw memory usage graphs, I see sawtooth waves to the memory usage which >> suggest that the garbage builds up until the GC kicks in and reaps the >> garbage. > > Interesting. How do you actually measure this memory usage? Often, > when a GC frees up memory, it's merely made available for subsequent > allocations, rather than actually given back to the system - all it > takes is one still-used object on a page and the whole page has to be > retained. > > As such, a "create and drop" usage model would tend to result in > memory usage going up for a while, but then remaining stable, as all > allocations are being fulfilled from previously-released memory that's > still owned by the process. > > > I'm measuring it using a bit of a hack; I use psutil.Popen > (https://pypi.python.org/pypi/psutil) to open a simulation as a child > process, and in a tight loop gather the size of the resident set and the > number of virtual pages currently in use of the child. The sawtooths are > about 10% (and decreasing) of the size of the overall memory usage, and are > probably due to different stages of the simulation doing different things. > That is an educated guess though, I don't have strong evidence to back it > up. > > And, yes, what you describe is pretty close to what I'm seeing. The longer > the simulation has been running, the smoother the memory usage gets. Ah, I think I understand. So the code would be something like this: Phase one: Create a bunch of objects Do a bunch of simulation Destroy a bunch of objects Simulate more Destroy all the objects used in this phase, other than the result Phase two: Like phase one In that case, yes, it's entirely possible that the end of a phase could signal a complete cleanup of intermediate state, with the consequent release of memory to the system. (Or, more likely, a near-complete cleanup, with release of MOST of memory.) Very cool bit of analysis you've done there. Thank you! And, yes, that is essentially what is going on (or was in that version of the simulator; I'm in the middle of a big refactor to speed things up and expect the memory usage patterns to change) Thanks, Cem Karan From steve+python at pearwood.info Thu Jun 22 16:58:55 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 23 Jun 2017 06:58:55 +1000 Subject: Progress on the Gilectomy References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <8c033515-9aaf-4ef7-ab3c-95d382b53710@googlegroups.com> <594af9f0$0$1618$c3e8da3$5496439d@news.astraweb.com> <3861efa9-ceaa-49f7-95b4-b38a0e14c9bc@googlegroups.com> Message-ID: <594c2f90$0$1602$c3e8da3$5496439d@news.astraweb.com> On Fri, 23 Jun 2017 01:07 am, breamoreboy at gmail.com wrote: > 11 comments on the thread "Instagram: 40% Py3 to 99% Py3 in 10 months" showing > that he knows as much about Unicode as LDO knows about garabge collection. Who cares? Every time he opens his mouth to write absolute rubbish he just makes a fool of himself. Why do you let it upset you? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jblack at nopam.com Thu Jun 22 17:08:51 2017 From: jblack at nopam.com (John Black) Date: Thu, 22 Jun 2017 16:08:51 -0500 Subject: comp.lang.python killfile rule Message-ID: All, in case this is useful to anyone, this rule that tells my newsreader which posts to kill really cleans up the group. I could not find a way to key off of anything in the header except keywords because the From keeps changing and I didn't want to overkill real posts. I may have to add a thing or two to this over time, but right now, this seems to be nailing everything. John Black Subject contains "PEDOFILO" Or Subject contains "MAI" Or Subject contains "SEGRETO" Or Subject contains "SETTA" Or Subject contains "BAMBINI" Or Subject contains "FIGLIO" Or Subject contains "PAOLO" Or Subject contains "NATALE" Or Subject contains "SONO" Or Subject contains "GRAZIA" Or Subject contains "PORNOSTAR" Or Subject contains "PEZZO" Or Subject contains "MERDA" Or Subject contains "CAZZO" Or Subject contains "GALERA" Or Subject contains "SICARIO" Or Subject contains "ESSERE" Or Subject contains "CRIMINALE" Or Subject contains "LECCA" Or Subject contains "COCAINA" Or Subject contains "LESBICA" Or Subject contains "NESSUNO" Or Subject contains "MAFIOSO" Or Subject contains "BERLUSCONI" Or Subject contains "????" Or Subject contains "HARDCORE" Or Subject contains "PEDERASTA" Or Subject contains "CULO" Or Subject contains "NOSTRA" Or Subject contains "FOGLIO" Or Subject contains "USARE" Or Subject contains "FAMIGLIA" Or Subject contains "FECE" Or Subject contains "CAPO" Or Subject contains "SUICIDARE" Or Subject contains "OGNI" Or Subject contains "CANE" Or Subject contains "MERCATO" Or Subject contains "VOLTA" Or Subject contains "MAFIOSA" Or Subject contains "ALMENO" Or Subject contains "BASTARDO" Or Subject contains "FIGLIA" Or Subject contains "BASTARD" Or Subject contains "CRIMINAL" Or Subject contains "ANNI" Or Subject contains "PEDINA" From marko at pacujo.net Thu Jun 22 17:21:23 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 23 Jun 2017 00:21:23 +0300 Subject: comp.lang.python killfile rule References: Message-ID: <87injn4t2k.fsf@elektro.pacujo.net> John Black : > All, in case this is useful to anyone, this rule that tells my newsreader > which posts to kill really cleans up the group. I could not find a way > to key off of anything in the header except keywords because the From > keeps changing and I didn't want to overkill real posts. I may have to > add a thing or two to this over time, but right now, this seems to be > nailing everything. I'm still wondering if those flurries of Italian calumnies are actually steganographic instructions to intelligence agents of some country. Marko From breamoreboy at gmail.com Thu Jun 22 17:53:07 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 22 Jun 2017 14:53:07 -0700 (PDT) Subject: [Distutils] ANNOUNCE: Sunsetting of uploading to the legacy PyPI/TestPyPI Message-ID: <25024729-180e-43db-8795-32bc6497bd4b@googlegroups.com> Things are moving on in the pypi world https://mail.python.org/pipermail/distutils-sig/2017-June/030766.html -- Kindest regards. Mark Lawrence From rosuav at gmail.com Thu Jun 22 17:56:48 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Jun 2017 07:56:48 +1000 Subject: comp.lang.python killfile rule In-Reply-To: <87injn4t2k.fsf@elektro.pacujo.net> References: <87injn4t2k.fsf@elektro.pacujo.net> Message-ID: On Fri, Jun 23, 2017 at 7:21 AM, Marko Rauhamaa wrote: > John Black : > >> All, in case this is useful to anyone, this rule that tells my newsreader >> which posts to kill really cleans up the group. I could not find a way >> to key off of anything in the header except keywords because the From >> keeps changing and I didn't want to overkill real posts. I may have to >> add a thing or two to this over time, but right now, this seems to be >> nailing everything. > > I'm still wondering if those flurries of Italian calumnies are actually > steganographic instructions to intelligence agents of some country. It's a super-sneaky code. You take all of the posts sent to c.l.p and determine whether they are spam or ham; a spam post represents a 0 bit, and a ham post is a 1 bit. The flurries of spam are carefully timed to interleave with actual user posts in order to transmit information. ChrisA From gulsumramazanoglu at gmail.com Thu Jun 22 18:41:43 2017 From: gulsumramazanoglu at gmail.com (gulsumramazanoglu at gmail.com) Date: Thu, 22 Jun 2017 15:41:43 -0700 (PDT) Subject: To connect to an AS400 over Python, do we have to pay for IBMs driver license? Message-ID: <04e62e7b-7d28-4ff6-8210-b1c51fdde628@googlegroups.com> Hi, I just wonder a point about connecting to an AS400 DB2 over Python and if you can shed some light, I will be happy... To connect to a DB2 of a remote AS400 over any one of the Python DB API interfaces (ibm_db, pyodbc, pyDB2 or any other one) from Python on Windows, in any case do we have to pay for the license of the IBM ODBC driver? I tried ibm_db and it seems its for free to connect to a DB2 on Windows, but due to pay for a license to connect to a remote AS400.. Is there any other way of connecting to a remote AS400 without IBM licensing "solutions", from Python3.x?... Thank you in advance.. From jladasky at itu.edu Thu Jun 22 20:01:58 2017 From: jladasky at itu.edu (jladasky at itu.edu) Date: Thu, 22 Jun 2017 17:01:58 -0700 (PDT) Subject: comp.lang.python killfile rule In-Reply-To: References: <87injn4t2k.fsf@elektro.pacujo.net> Message-ID: On Thursday, June 22, 2017 at 2:57:02 PM UTC-7, Chris Angelico wrote: > On Fri, Jun 23, 2017 at 7:21 AM, Marko Rauhamaa wrote: > > I'm still wondering if those flurries of Italian calumnies are actually > > steganographic instructions to intelligence agents of some country. > > It's a super-sneaky code. You take all of the posts sent to c.l.p and > determine whether they are spam or ham; a spam post represents a 0 > bit, and a ham post is a 1 bit. The flurries of spam are carefully > timed to interleave with actual user posts in order to transmit > information. I've wanted to read "Mother Night" by Kurt Vonnegut for quite come time. From rustompmody at gmail.com Fri Jun 23 01:26:34 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 22 Jun 2017 22:26:34 -0700 (PDT) Subject: Progress on the Gilectomy In-Reply-To: <594af9f0$0$1618$c3e8da3$5496439d@news.astraweb.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <87poe03k1f.fsf@nightsong.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <8c033515-9aaf-4ef7-ab3c-95d382b53710@googlegroups.com> <594af9f0$0$1618$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thursday, June 22, 2017 at 4:28:03 AM UTC+5:30, Steve D'Aprano wrote: > On Thu, 22 Jun 2017 08:23 am, breamoreboy wrote: > > > Don't you know that Lawrence D?Oliveiro has been banned from the mailing list > > as he hasn't got a clue what he's talking about, > > That's not why he was given a ban. Being ignorant is not a crime -- if it were, > a lot more of us would be banned, including all newbies. Lawrence d'Oliveiro was banned on 30th Sept 2016 till end-of-year https://mail.python.org/pipermail/python-list/2016-September/714725.html Is there still a ban? From greg.ewing at canterbury.ac.nz Fri Jun 23 01:44:05 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 23 Jun 2017 17:44:05 +1200 Subject: Best way to ensure user calls methods in correct order? In-Reply-To: <594bd6d3$0$1589$c3e8da3$5496439d@news.astraweb.com> References: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> <594bd6d3$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > There are a very few exceptions to this rule of thumb, such as opening > connections to databases or files or similar. Another way to handle that is to have the connection method return another object that has the methods that should only be called for active connections. That way it's not possible to do things out of sequence. -- Greg From greg.ewing at canterbury.ac.nz Fri Jun 23 01:48:23 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 23 Jun 2017 17:48:23 +1200 Subject: Progress on the Gilectomy In-Reply-To: <87o9tgvwzc.fsf@elektro.pacujo.net> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <8760foxid0.fsf@elektro.pacujo.net> <87wp84vx9b.fsf@elektro.pacujo.net> <87o9tgvwzc.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > And, BTW, my rule of thumb came from experiences with the Hotspot JRE. I wouldn't take a Java implementation to be representative of the behaviour of GC systems in general. -- Greg From greg.ewing at canterbury.ac.nz Fri Jun 23 01:53:16 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 23 Jun 2017 17:53:16 +1200 Subject: comp.lang.python killfile rule In-Reply-To: <87injn4t2k.fsf@elektro.pacujo.net> References: <87injn4t2k.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > I'm still wondering if those flurries of Italian calumnies are actually > steganographic instructions to intelligence agents of some country. Or a C&C system for someone's botnet? -- Greg From jussi.piitulainen at helsinki.fi Fri Jun 23 02:49:06 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Fri, 23 Jun 2017 09:49:06 +0300 Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency References: <1qtnkcp6k42qg2peg4174shsv0c2a4bg8a@4ax.com> <8lkokcp8peo5sh8q2t2nnrrj2hrs3pdepr@4ax.com> Message-ID: Dennis Lee Bieber writes: > On Thu, 22 Jun 2017 22:46:28 +0300, Jussi Piitulainen declaimed the > following: > >> >> A pair of methods, str.maketrans to make a translation table and then >> .translate on every string, allows to do all that in one step: >> >> spacy = r'\/-.[]{}()' >> tr = str.maketrans(dict.fromkeys(spacy, ' ')) >> >> ... >> >> ln = ln.translate(tr) >> >> But those seem to be only in Python 3. >> > > Well -- I wasn't trying for "production ready" either; mostly > focusing on the SQLite side of things. I know, and that's a sound suggestion if the OP is ready for that. I just like those character translation methods, and I didn't like it when you first took the time to call a simple regex "line noise" and then proceeded to post something that looked much more noisy yourself. >> However, if the OP really is getting their input from a CSV file, >> they shouldn't need methods like these. Because surely it's then >> already an unambiguous list of words, to be read in with the csv >> module? Or else it's not yet CSV at all after all? I think they need >> to sit down with someone who can walk them through the whole >> exercise. > > The OP file extensions had CSV, but there was no sign of the csv > module being used; worse, it looks like the write of the results file > has no formatting -- it is the repr of a tuple of (word, count)! Exactly. Too many things like that makes me think they are not ready for more advanced methods. > I'm going out on a limb and guessing the regex being used to > find words is accepting anything separated by leading/trailing space > containing a minimum of 3 and maximum of 15 characters in the set > a..z. So could be missing first and last words on a line if they don't > have the leading or trailing space, and ignoring "a", "an", "me", > etc., along with "mrs." [due to .] In contrast, I didn't limit on > length, and tried to split "look-alike" into "look" and "alike" (and > given time, would have tried to accept "people's" as a possessive). I'm not sure I like the splitting of look-alike (I'm not sure that I like not splitting it either) but note that the regex does that for free. The \b in the original regex matches the empty string at a position where there is a "word character" on only one side. It recognizes a boundary at the beginning of a line and at whitespace, but also at all the punctuation marks. You guess right about the length limits. I wouldn't use them, and then there's no need for the boundary markers any more: my \w+ matches maximal sequences of word characters (even in foreign languages like Finnish or French, and even in upper case, also digits). To also match "people's" and "didn't", use \w+'\w+, and to match with and without the ' make the trailing part optional \w+('\w+)? except the notation really does start to become noisy because one must prevent the parentheses from "capturing" the group: import re wordy = re.compile(r''' \w+ (?: ' \w+ )? ''', re.VERBOSE) text = ''' Oliver N'Goma, dit Noli, n? le 23 mars 1959 ? Mayumba et mort le 7 juin 2010, est un chanteur et guitariste gabonais d'Afro-zouk. ''' print(wordy.findall(text)) # ['Oliver', "N'Goma", 'dit', 'Noli', 'n?', 'le', '23', 'mars', '1959', # '?', 'Mayumba', 'et', 'mort', 'le', '7', 'juin', '2010', 'est', 'un', # 'chanteur', 'et', 'guitariste', 'gabonais', "d'Afro", 'zouk'] Not too bad? But some punctuation really belongs in words. And other doesn't. And everything becomes hard and every new heuristic turns out too strict or too lenient and things that are not words at all may look like words, or it may not be clear whether something is a word or is more than one word or is less than a word or not like a word at all. Should one be amused? Should one despair? :) From greg.ewing at canterbury.ac.nz Fri Jun 23 04:07:29 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 23 Jun 2017 20:07:29 +1200 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) In-Reply-To: <2ce6fef6-4552-47b5-b3b1-38ade5514278@googlegroups.com> References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <8760foxid0.fsf@elektro.pacujo.net> <87wp84vx9b.fsf@elektro.pacujo.net> <87o9tgvwzc.fsf@elektro.pacujo.net> <2ce6fef6-4552-47b5-b3b1-38ade5514278@googlegroups.com> Message-ID: Lawrence D?Oliveiro wrote: > what WOULD you consider to be so ?representative?? I don't claim any of them to be representative. Different GC strategies have different characteristics. -- Greg From marko at pacujo.net Fri Jun 23 05:04:16 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 23 Jun 2017 12:04:16 +0300 Subject: Progress on the Gilectomy (Posting On Python-List Prohibited) References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <8760foxid0.fsf@elektro.pacujo.net> <87wp84vx9b.fsf@elektro.pacujo.net> <87o9tgvwzc.fsf@elektro.pacujo.net> <2ce6fef6-4552-47b5-b3b1-38ade5514278@googlegroups.com> Message-ID: <87tw372hyn.fsf@elektro.pacujo.net> Gregory Ewing : > Lawrence D?Oliveiro wrote: >> what WOULD you consider to be so ?representative?? > > I don't claim any of them to be representative. Different GC > strategies have different characteristics. My experiences with Hotspot were a bit disheartening. GC is a winning concept provided that you don't have to strategize too much. In practice, it seems tweaking the GC parameters is a frequent necessity. On the other hand, I believe much of the trouble comes from storing too much information in the heap. Applications shouldn't have semipersistent multigigabyte lookup structures kept in RAM, at least not in numerous small objects. Marko From mal at europython.eu Fri Jun 23 06:33:04 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Fri, 23 Jun 2017 12:33:04 +0200 Subject: PyData EuroPython 2017 Message-ID: <935b3189-bf07-1371-0e2d-02924258d5bd@europython.eu> We are excited to announce a complete PyData track at EuroPython 2017 in Rimini, Italy from the 9th to 16th July. * PyData EuroPython 2017 * https://ep2017.europython.eu/en/events/pydata/ The PyData track will be part of EuroPython 2017, so you won?t need to buy an extra ticket to attend. Mostly talks and trainings are scheduled for Wednesday, Thursday and Friday (July 12-14), with a few on other days as well. We will have over 40 talks, 5 trainings, and 2 keynotes dedicated to PyData. If you?d like to attend PyData EuroPython 2017, please register for EuroPython 2017 soon: https://ep2017.europython.eu/en/registration/ Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/878198420573483008 Thanks. From ethan at stoneleaf.us Fri Jun 23 10:36:05 2017 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 23 Jun 2017 07:36:05 -0700 Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <8c033515-9aaf-4ef7-ab3c-95d382b53710@googlegroups.com> <594af9f0$0$1618$c3e8da3$5496439d@news.astraweb.com> Message-ID: <594D2755.7010109@stoneleaf.us> On 06/22/2017 10:26 PM, Rustom Mody wrote: > Lawrence d'Oliveiro was banned on 30th Sept 2016 till end-of-year > https://mail.python.org/pipermail/python-list/2016-September/714725.html > > Is there still a ban? My apologies to Lawrence, I completely forgot. The ban is now lifted. -- ~Ethan~ From __peter__ at web.de Fri Jun 23 12:13:25 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 23 Jun 2017 18:13:25 +0200 Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency References: <1qtnkcp6k42qg2peg4174shsv0c2a4bg8a@4ax.com> <8lkokcp8peo5sh8q2t2nnrrj2hrs3pdepr@4ax.com> Message-ID: Dennis Lee Bieber wrote: > On Fri, 23 Jun 2017 09:49:06 +0300, Jussi Piitulainen > declaimed the following: > >>I just like those character translation methods, and I didn't like it >>when you first took the time to call a simple regex "line noise" and >>then proceeded to post something that looked much more noisy yourself. >> > > Tediously long (and likely slow running), but I'd think each .replace() > would have been self-explanatory. > >>I'm not sure I like the splitting of look-alike (I'm not sure that I >>like not splitting it either) but note that the regex does that for >>free. >> >>The \b in the original regex matches the empty string at a position >>where there is a "word character" on only one side. It recognizes a >>boundary at the beginning of a line and at whitespace, but also at all >>the punctuation marks. >> >>You guess right about the length limits. I wouldn't use them, and then >>there's no need for the boundary markers any more: my \w+ matches >>maximal sequences of word characters (even in foreign languages like >>Finnish or French, and even in upper case, also digits). >> >>To also match "people's" and "didn't", use \w+'\w+, and to match with >>and without the ' make the trailing part optional \w+('\w+)? except the >>notation really does start to become noisy because one must prevent the >>parentheses from "capturing" the group: >> >>import re >>wordy = re.compile(r''' \w+ (?: ' \w+ )? ''', re.VERBOSE) >>text = ''' >>Oliver N'Goma, dit Noli, n? le 23 mars 1959 ? Mayumba et mort le 7 juin >>2010, est un chanteur et guitariste gabonais d'Afro-zouk. >>''' >> >>print(wordy.findall(text)) >> >># ['Oliver', "N'Goma", 'dit', 'Noli', 'n?', 'le', '23', 'mars', '1959', >># '?', 'Mayumba', 'et', 'mort', 'le', '7', 'juin', '2010', 'est', 'un', >># 'chanteur', 'et', 'guitariste', 'gabonais', "d'Afro", 'zouk'] >> >>Not too bad? > > Above content saved (in a write-only file? I don't recall the times > I've searched my post archives) for potential future use. I should plug it > into my demo and see how much speed improvement I get. Most of the potential speedup can be gained from using collections.Counter() instead of the database. If necessary write the counter's contents into the database in a second step. From bill.janssen at gmail.com Fri Jun 23 12:58:32 2017 From: bill.janssen at gmail.com (bill.janssen at gmail.com) Date: Fri, 23 Jun 2017 09:58:32 -0700 (PDT) Subject: matplotlib change? In-Reply-To: References: <5f81ee9e-8ac1-480f-8110-ae129f3837fd@googlegroups.com> <44af14a2-d4c0-4a49-a0d4-652375e36070@googlegroups.com> Message-ID: On Thursday, June 22, 2017 at 11:24:53 AM UTC-7, Michael F. Stemper wrote: > On 2017-06-22 12:56, bill.janssen at gmail.com wrote: > > On Thursday, June 22, 2017 at 10:14:21 AM UTC-7, Michael F. Stemper wrote: > >> On 2017-06-22 09:50, breamoreboy at gmail.com wrote: > >>> On Thursday, June 22, 2017 at 3:33:36 PM UTC+1, Michael F. Stemper wrote: > > > > >>>> Is it likely that the difference in plots due to something that > >>>> changed in matplotlib between 2.7.12 and 2.7.13? If so, is there > >>>> some argument that I could specify in one of the functions to > >>>> prevent this padding/margin/waste? Is there a separate function > >>>> to call? > >>>> > >>>> If the difference isn't due to a change in matplotlib, would it be > >>>> something OS-dependent? How can I track it down? > >> > >>> Id check to see which matplotlib versions you have rather than the Python version. Either:- > >> > >> Okay, that was easy enough: > > >> >>> import matplotlib > >> >>> matplotlib.__version__ > >> '2.0.0' > >> >>> > >> > >> What's my next step? > > > There were a number of changes to line-drawing in the bump from 1 to 2 -- I found that some of them were not listed in the change notes. It's worth reviewing https://matplotlib.org/users/dflt_style_changes.html#plotting-functions. > > That nailed it! This section: > > addressed my exact problem. > > (Unfortunately, the parameters are not back-compatible, but at least > I'll be ready when they upgrade the box where I usually work.) > > Thanks for your help. > > BTW, your name is familiar. Were you ever at PARC? > > -- > Michael F. Stemper > This post contains greater than 95% post-consumer bytes by weight. Still am. Bill From neilc at norwich.edu Fri Jun 23 14:28:17 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Fri, 23 Jun 2017 18:28:17 +0000 (UTC) Subject: Best way to ensure user calls methods in correct order? References: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> <594bd6d3$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2017-06-23, Gregory Ewing wrote: > Steve D'Aprano wrote: >> There are a very few exceptions to this rule of thumb, such as >> opening connections to databases or files or similar. > > Another way to handle that is to have the connection method > return another object that has the methods that should only be > called for active connections. That way it's not possible to do > things out of sequence. It's like a bidirectional iterator in C++, except in reverse it's random access. An iterator that can't easily be modeled with a generator in Python is going to feel awkward. -- Neil Cerutti From mbyrnepr2 at gmail.com Fri Jun 23 16:07:15 2017 From: mbyrnepr2 at gmail.com (Mark Byrne) Date: Fri, 23 Jun 2017 21:07:15 +0100 Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency Message-ID: A problem (possibly the problem) is the lines which use the get function: count = frequency.get(word,0) Since the dictionary is empty at the start of the loop, the get function is passing a value of 0 to count and count1. The subsequent updates to the dictionary are applying a value of zero As a result, the file being written to contains count values of 0 for each word. Possible fix is to replace this: count = frequency.get(word,0) count1 = frequency.get(word1,0) if word1 == word: frequency[word] = count + count1 else: frequency[word] = count with this: if word1 == word: if word in frequency: frequency[word] += 1 else: frequency[word] = 1 From mbyrnepr2 at gmail.com Fri Jun 23 20:04:04 2017 From: mbyrnepr2 at gmail.com (mbyrnepr2 at gmail.com) Date: Fri, 23 Jun 2017 17:04:04 -0700 (PDT) Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency In-Reply-To: References: Message-ID: <37e30896-8348-4ad3-a94a-7b436d59603c@googlegroups.com> On Thursday, June 22, 2017 at 12:16:28 PM UTC+1, kishan.samp... at gmail.com wrote: > I want to write a common file in which It can add the frequency by adding multiple csv file and if the same words are repeated in python then it should add the frequency in the common file can any one help me please > > > import re > import operator > import string > > class words: > def __init__(self,fh): > self.fh = fh > def read(self): > for line in fh: > yield line.split() > > if __name__ == "__main__": > frequency = {} > document_text = open('data_analysis.csv', 'r') > common1_file = open("common_file1.csv", "r") > > text_string = document_text.read().lower() > match_pattern = re.findall(r'\b[a-z]{3,15}\b', text_string) > > text_string_one = common1_file.read().lower() > match_pattern_one = re.findall(r'\b[a-z]{3,15}\b', text_string_one) > #print("match_pattern"+(str(match_pattern))) > for word in match_pattern: > for word1 in match_pattern_one: > count = frequency.get(word,0) > count1 = frequency.get(word1,0) > if word1 == word: > frequency[word] = count + count1 > else: > frequency[word] = count > > > frequency_list = frequency.keys() > text_file = open("common_file1.csv", "w") > for words in frequency_list: > data = (words, frequency[words]) > print (data) > #text_file = open("common_file1.csv", "w") > #for i in data: > #store_fre = (str(data)+"\n") > text_file.write(str(data)+"\n") > > > text_file.close() > > > this is my code written by me til now but not getting satisfied results Dictionary 'frequency' is updated only with values of 0. If the aim is to get a count of occurrences for each word where the word exists in both input files, you could replace this: for word in match_pattern: for word1 in match_pattern_one: count = frequency.get(word,0) count1 = frequency.get(word1,0) if word1 == word: frequency[word] = count + count1 else: frequency[word] = count with this: all_words = match_pattern + match_pattern_one word_set = set(match_pattern) & set(match_pattern_one) while word_set: word = word_set.pop() count = all_words.count(word) frequency[word] = count Other observations: - Reading from and writing to the csv files is not utilsing the csv format - The regex may be too restrictive and not all expected words extracted - The output is written to one of the input files, overwriting the original content of the input file From steve+python at pearwood.info Sat Jun 24 02:20:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 24 Jun 2017 16:20:38 +1000 Subject: Deleting An Optional Dict Entry (Posting On Python-List Prohibited) References: <062d9779-42e8-4348-b59c-0eac1716490c@googlegroups.com> Message-ID: <594e04b8$0$1601$c3e8da3$5496439d@news.astraweb.com> On Fri, 23 Jun 2017 07:19 pm, Lawrence D?Oliveiro wrote: > The construct > > del ?dict?[?key?] > > deletes the entry with key ?key? from the dictionary ?dict?, provided it > exists. If it does not, you get a KeyError. > > To delete an entry which might or might not exist, the simplest way is this: > > ?dict?.pop(?key?, None) > > Actually, the second argument to the pop method doesn?t matter, since the > result from the call is ignored in this case. Nice hint. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From dhariyalbhaskar at gmail.com Sat Jun 24 05:32:08 2017 From: dhariyalbhaskar at gmail.com (Bhaskar Dhariyal) Date: Sat, 24 Jun 2017 02:32:08 -0700 (PDT) Subject: Unable to convert pandas object to string Message-ID: <6ed7b0a1-53bc-437f-be7b-0266090dc9a5@googlegroups.com> Int64Index: 171594 entries, 0 to 63464 Data columns (total 7 columns): project_id 171594 non-null object desc 171594 non-null object goal 171594 non-null float64 keywords 171594 non-null object diff_creat_laun 171594 non-null int64 diff_laun_status 171594 non-null int64 diff_status_dead 171594 non-null int64 dtypes: float64(1), int64(3), object(3) not able to convert desc and keywords to string for preprocessing. Tried astype(str). Please help From paul.james.barry at gmail.com Sat Jun 24 05:44:54 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Sat, 24 Jun 2017 10:44:54 +0100 Subject: Unable to convert pandas object to string In-Reply-To: <6ed7b0a1-53bc-437f-be7b-0266090dc9a5@googlegroups.com> References: <6ed7b0a1-53bc-437f-be7b-0266090dc9a5@googlegroups.com> Message-ID: Any chance you could post one line of data so we can see what we have to work with? Also - have you taken a look at Jake VanderPlas's notebooks? There's lot of help with pandas to be found there: https://github.com/jakevdp/PythonDataScienceHandbook Paul. On 24 June 2017 at 10:32, Bhaskar Dhariyal wrote: > > Int64Index: 171594 entries, 0 to 63464 > Data columns (total 7 columns): > project_id 171594 non-null object > desc 171594 non-null object > goal 171594 non-null float64 > keywords 171594 non-null object > diff_creat_laun 171594 non-null int64 > diff_laun_status 171594 non-null int64 > diff_status_dead 171594 non-null int64 > dtypes: float64(1), int64(3), object(3) > > not able to convert desc and keywords to string for preprocessing. > Tried astype(str). Please help > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From tjol at tjol.eu Sat Jun 24 05:53:45 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 24 Jun 2017 11:53:45 +0200 Subject: Best way to ensure user calls methods in correct order? In-Reply-To: References: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> <594bd6d3$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: <594e375b$0$1694$e4fe514c@news.kpn.nl> On 22/06/17 17:31, Thomas Nyberg wrote: > Thanks for the response! I see the reasoning, but I can't entirely > square it with what I'm thinking. Probably it's either because I was > hiding some of my intention (sorry if I wasted your time due to lack of > details) or that I'm naive and regardless of intention doing something > silly. > > On 06/22/2017 04:40 PM, Steve D'Aprano wrote: >> On Thu, 22 Jun 2017 11:53 pm, Thomas Nyberg wrote: >> >> Don't do that. It's fragile and an anti-pattern. Your methods have too much >> coupling. If c() relies on b() being called first, then either b() or c() >> aren't good methods. They don't do enough: >> >> - calling b() alone doesn't do enough, so you have to call c() next to get the >> job done; >> >> - calling c() alone doesn't do enough, because it relies on b() being called >> first. >> >> >> There are a very few exceptions to this rule of thumb, such as opening >> connections to databases or files or similar. They are mostly enshrined from >> long practice, or justified by low-level APIs (that's how file systems and >> databases work, and it's not practical to change that). But you should try very >> hard to avoid creating new examples. >> >> Of course, I'm talking about your *public* methods. Private methods can be a bit >> more restrictive, since it's only *you* who suffers if you do it wrong. >> >> Ideally, your methods should be written in a functional style with as little >> shared state as possible. (Shared state introduces coupling between components, >> and excessive coupling is *the* ultimate evil in programming.) >> > > This makes perfect sense in general. An important detail I left out > earlier was what exactly I meant by "users" of my code. Really all I > meant was myself calling this code as a library and avoiding making > mistakes. Currently what I'm doing is kind of a data storage, data > processing/conversion, data generation pipeline. This different > variables at the different steps in the pipeline are specified in a > script. The functions in this script have _no coupling at all_ for all > the reasons you stated. In fact, the only reason I'm introducing a class > is because I'm trying to force their ordering in the way I described. > > The ultimate goal here is to instead put a basic http server wrapper > around the whole process. The reason I want to do this is to allow > others (internal to the company) to make changes in the process > interactively at different steps in the process (the steps are "natural" > for the problem at hand) without editing the python scripts explicitly. > > Of course I could force them to execute everything in one go, but due to > the time required for the different steps, it's nicer to allow for some > eye-balling of results (and possible changes and re-running) before > continuing. Before this the main way of enforcing this was all done by > the few of use editing the scripts, but now I'd like to make it an error > for corrupted data to propagate down through the pipeline. Having a > run_all() method or something similar will definitely exist (usually > once you figure out the "right" parameters you don't change them much on > re-runs), but having things broken apart is still very nice. > > Would you still say this is a bad way of going about it? Thanks for the > feedback. It's very helpful. If a() does some processing, and then b() does something else to the result of a(), then the natural way of calling the functions is probably c(b(a(initial_data))), rather than a sequence of method or function calls that hide some internal state. If the user wants to jump in and look at what's going on, they can: a_result = a() # peruse the data b_result = b(a_result) # have some coffee c_result = c(b_result) # and so on. If the data is modified in-place, maybe it makes sense to to use a class like the one you have, but then it's a bit bizarre to make your methods create an artificial side effect (self._a_dirty) - why don't you simply check for the actual effect of a(), whatever it is. If a() did some calculations and added a column to your dataset with the results, check for the existence of that column in b()! If calling b() invalidates some calculation results generated in c(), delete them! (In the functional setup above, b() would refuse to process a dataset that it has already processed) -- Thomas From paul.james.barry at gmail.com Sat Jun 24 06:24:11 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Sat, 24 Jun 2017 11:24:11 +0100 Subject: Unable to convert pandas object to string In-Reply-To: References: <6ed7b0a1-53bc-437f-be7b-0266090dc9a5@googlegroups.com> Message-ID: Hi Bhaskar. Please see attached PDF of a small Jupyter notebook. As you'll see, the data in the fields you mentioned are *already* strings. What is it you are trying to do here? Paul. On 24 June 2017 at 10:51, Bhaskar Dhariyal wrote: > ? > train.csv > > ?here it is thanks for quick reply > > On Sat, Jun 24, 2017 at 3:14 PM, Paul Barry > wrote: > >> Any chance you could post one line of data so we can see what we have to >> work with? >> >> Also - have you taken a look at Jake VanderPlas's notebooks? There's lot >> of help with pandas to be found there: https://github.com/jake >> vdp/PythonDataScienceHandbook >> >> Paul. >> >> On 24 June 2017 at 10:32, Bhaskar Dhariyal >> wrote: >> >>> >>> Int64Index: 171594 entries, 0 to 63464 >>> Data columns (total 7 columns): >>> project_id 171594 non-null object >>> desc 171594 non-null object >>> goal 171594 non-null float64 >>> keywords 171594 non-null object >>> diff_creat_laun 171594 non-null int64 >>> diff_laun_status 171594 non-null int64 >>> diff_status_dead 171594 non-null int64 >>> dtypes: float64(1), int64(3), object(3) >>> >>> not able to convert desc and keywords to string for preprocessing. >>> Tried astype(str). Please help >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> >> >> >> >> -- >> Paul Barry, t: @barrypj - w: >> http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >> > > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From steve+python at pearwood.info Sat Jun 24 06:31:01 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 24 Jun 2017 20:31:01 +1000 Subject: Checking for an exception Message-ID: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> What's the right/best way to test whether an object is an exception ahead of time? (That is, without trying to raise from it.) I have: return (isinstance(obj, type) and issubclass(obj, BaseException) or isinstance(obj, BaseException)) Any better ideas? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From mbyrnepr2 at gmail.com Sat Jun 24 07:23:46 2017 From: mbyrnepr2 at gmail.com (mbyrnepr2 at gmail.com) Date: Sat, 24 Jun 2017 04:23:46 -0700 (PDT) Subject: Checking for an exception In-Reply-To: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: <977c8d4c-9c5f-4db3-ad24-61e4dc32b631@googlegroups.com> On Saturday, June 24, 2017 at 11:31:11 AM UTC+1, Steve D'Aprano wrote: > What's the right/best way to test whether an object is an exception ahead of > time? (That is, without trying to raise from it.) > > I have: > > return (isinstance(obj, type) and issubclass(obj, BaseException) > or isinstance(obj, BaseException)) > > > Any better ideas? > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. Would something along these lines help? import exceptions if 'ArithmeticError' in dir(exceptions): print 'is an exception' try: if exceptions.__getattribute__('ArithmeticError'): print 'is an exception' except AttributeError: print 'is not an exception' try: getattr(exceptions, 'ArithmeticError') print 'is not an exception' except AttributeError: print 'is not an exception' From paul.james.barry at gmail.com Sat Jun 24 07:31:48 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Sat, 24 Jun 2017 12:31:48 +0100 Subject: Fwd: Unable to convert pandas object to string In-Reply-To: References: <6ed7b0a1-53bc-437f-be7b-0266090dc9a5@googlegroups.com> Message-ID: Forgot to include this reply to the list (as others may want to comment). ---------- Forwarded message ---------- From: Paul Barry Date: 24 June 2017 at 12:21 Subject: Re: Unable to convert pandas object to string To: Bhaskar Dhariyal Note that .info(), according to its docs, gives you a "Concise summary of a DataFrame". Everything is an object in Python, including strings, so the output from .info() is technically correct (but maybe not very helpful in your case). As I've shown, we can work out that the data you want to work with is in fact a string, so I've added some code to my notebook to show you how to tokenize the first row of data. This should get you started on doing this to the rest of your data. Note, too, that some of the data in these specific columns contains something other than a string, so you'll need to clean up that first (see the end of the updated notebook, attached, for how I worked out that this was indeed the case). I hope this all helps. Paul. On 24 June 2017 at 11:31, Bhaskar Dhariyal wrote: > The data type showing there is object. In[4] in the first page. I wanted > to tokenize the name & desc column and clean it > > > On Sat, Jun 24, 2017 at 3:54 PM, Paul Barry > wrote: > >> Hi Bhaskar. >> >> Please see attached PDF of a small Jupyter notebook. As you'll see, the >> data in the fields you mentioned are *already* strings. What is it you are >> trying to do here? >> >> Paul. >> >> On 24 June 2017 at 10:51, Bhaskar Dhariyal >> wrote: >> >>> ? >>> train.csv >>> >>> ?here it is thanks for quick reply >>> >>> On Sat, Jun 24, 2017 at 3:14 PM, Paul Barry >>> wrote: >>> >>>> Any chance you could post one line of data so we can see what we have >>>> to work with? >>>> >>>> Also - have you taken a look at Jake VanderPlas's notebooks? There's >>>> lot of help with pandas to be found there: https://github.com/jake >>>> vdp/PythonDataScienceHandbook >>>> >>>> Paul. >>>> >>>> On 24 June 2017 at 10:32, Bhaskar Dhariyal >>>> wrote: >>>> >>>>> >>>>> Int64Index: 171594 entries, 0 to 63464 >>>>> Data columns (total 7 columns): >>>>> project_id 171594 non-null object >>>>> desc 171594 non-null object >>>>> goal 171594 non-null float64 >>>>> keywords 171594 non-null object >>>>> diff_creat_laun 171594 non-null int64 >>>>> diff_laun_status 171594 non-null int64 >>>>> diff_status_dead 171594 non-null int64 >>>>> dtypes: float64(1), int64(3), object(3) >>>>> >>>>> not able to convert desc and keywords to string for preprocessing. >>>>> Tried astype(str). Please help >>>>> -- >>>>> https://mail.python.org/mailman/listinfo/python-list >>>>> >>>> >>>> >>>> >>>> -- >>>> Paul Barry, t: @barrypj - w: >>>> http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >>>> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >>>> >>> >>> >> >> >> -- >> Paul Barry, t: @barrypj - w: >> http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >> > > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From steve+python at pearwood.info Sat Jun 24 07:48:37 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 24 Jun 2017 21:48:37 +1000 Subject: Checking for an exception References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> <977c8d4c-9c5f-4db3-ad24-61e4dc32b631@googlegroups.com> Message-ID: <594e5198$0$1599$c3e8da3$5496439d@news.astraweb.com> On Sat, 24 Jun 2017 09:23 pm, mbyrnepr2 at gmail.com wrote: > On Saturday, June 24, 2017 at 11:31:11 AM UTC+1, Steve D'Aprano wrote: >> What's the right/best way to test whether an object is an exception ahead of >> time? (That is, without trying to raise from it.) >> >> I have: >> >> return (isinstance(obj, type) and issubclass(obj, BaseException) >> or isinstance(obj, BaseException)) >> >> >> Any better ideas? > Would something along these lines help? > > import exceptions py> import exceptions Traceback (most recent call last): File "", line 1, in ImportError: No module named 'exceptions' What module are you importing? > if 'ArithmeticError' in dir(exceptions): > print 'is an exception' I don't understand that. I'm not looking for strings which happen to match the names of built-in exceptions, I'm looking to test for any object which is an exception. To be an exception, you must inherit from BaseException. (String exceptions did work until Python 2.5, but I don't care about them.) > try: > if exceptions.__getattribute__('ArithmeticError'): You shouldn't call __getattribute__ directly -- that's reserved for Python's use. You should call getattr() with an optional default value: if getattr(exceptions, 'ArithmeticError', False): print 'is an exception' else: print 'is an exception' But in any case, no, I don't want to look up a string in some list of exceptions. That won't test for exceptions that aren't in the list. class MyCustomException(LookupError): pass -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tomuxiong at gmx.com Sat Jun 24 09:02:52 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Sat, 24 Jun 2017 15:02:52 +0200 Subject: Best way to ensure user calls methods in correct order? In-Reply-To: <594e375b$0$1694$e4fe514c@news.kpn.nl> References: <6cb78693-2e3b-5b6f-fe7b-b1455e9e048c@gmx.com> <594bd6d3$0$1589$c3e8da3$5496439d@news.astraweb.com> <594e375b$0$1694$e4fe514c@news.kpn.nl> Message-ID: On 06/24/2017 11:53 AM, Thomas Jollans wrote: > If the data is modified in-place, maybe it makes sense to to use a class > like the one you have, but then it's a bit bizarre to make your methods > create an artificial side effect (self._a_dirty) - why don't you simply > check for the actual effect of a(), whatever it is. If a() did some > calculations and added a column to your dataset with the results, check > for the existence of that column in b()! If calling b() invalidates some > calculation results generated in c(), delete them! (In the functional > setup above, b() would refuse to process a dataset that it has already > processed) > > -- Thomas > > Thanks I like this a lot! This does fit in well with my setup and I think it probably is the cleanest approach. It also makes it pretty obvious to tell where in the process you are by just looking at the data (though this is more of a convenience for myself than anyone else since the internal steps are hidden). Thanks a lot! Cheers, Thomas From sjeik_appie at hotmail.com Sat Jun 24 09:14:42 2017 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Sat, 24 Jun 2017 13:14:42 +0000 Subject: Fw: Unable to convert pandas object to string In-Reply-To: References: <6ed7b0a1-53bc-437f-be7b-0266090dc9a5@googlegroups.com>, , Message-ID: ________________________________ From: Albert-Jan Roskam Sent: Saturday, June 24, 2017 11:26:26 AM To: Paul Barry Subject: Re: Unable to convert pandas object to string (sorry for top posting) Try using fillna('') to convert np.nan into empty strings. df['desc'] = df.desc.fillna(''). Btw, np.object already is what best approximates str. I wish np.object had its own sentinel value for missing data instead of np.nan, which is a float. ________________________________ From: Python-list on behalf of Paul Barry Sent: Saturday, June 24, 2017 9:44:54 AM To: Bhaskar Dhariyal Cc: python-list at python.org Subject: Re: Unable to convert pandas object to string Any chance you could post one line of data so we can see what we have to work with? Also - have you taken a look at Jake VanderPlas's notebooks? There's lot of help with pandas to be found there: https://github.com/jakevdp/PythonDataScienceHandbook Paul. On 24 June 2017 at 10:32, Bhaskar Dhariyal wrote: > > Int64Index: 171594 entries, 0 to 63464 > Data columns (total 7 columns): > project_id 171594 non-null object > desc 171594 non-null object > goal 171594 non-null float64 > keywords 171594 non-null object > diff_creat_laun 171594 non-null int64 > diff_laun_status 171594 non-null int64 > diff_status_dead 171594 non-null int64 > dtypes: float64(1), int64(3), object(3) > > not able to convert desc and keywords to string for preprocessing. > Tried astype(str). Please help > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. -- https://mail.python.org/mailman/listinfo/python-list From j.clarke.873638 at gmail.com Sat Jun 24 09:40:23 2017 From: j.clarke.873638 at gmail.com (J. Clarke) Date: Sat, 24 Jun 2017 09:40:23 -0400 Subject: New to Python - Career question References: <30b628b0-80a2-42a3-a11c-6da05f64d965@googlegroups.com> <87efuwzqrq.fsf@elektro.pacujo.net> Message-ID: In article , larry.martell at gmail.com says... > > On Tue, Jun 6, 2017 at 6:37 PM, Marko Rauhamaa wrote: > > ptanyc at gmail.com: > > > >> New to Python and have been at it for about a month now. I'm doing > >> well and like it very much. Considering a career change down the road > >> and have been wondering... What are the job prospects for a middle age > >> entry level programmer. Just trying to get a better understanding > >> where I stand career wise. Appreciate all feed back. Thanks! > > > > Different employers hire differently. I have hired several people for my > > employer, and age has never been a concern. Python is also an important > > tool where I work. > > > > However, the problem in our field is that you have to be quite good to > > be truly useful. Unfortunately, it seems that only a minority with a > > formal degree are good enough. On the other hand, I work with some great > > software developers who don't have a degree at all. > > > > One good way to become a good developer and also test oneself is to pick > > a free software project online a become a contributor. Your commit log > > entries on GitHub advertise you much better than any pretty-printed > > r?sum?. > > I work 70 or more hours a week and have a family and hobbies and a > personal life. I do not have time to contribute to any open source > projects. And because I cannot show anything on GitHub I have often > been summarily disqualified for even getting an interview. (I have > donated money to open source projects.) If you're looking to change careers to programmer and you have hobbies that don't include programming, reconsider the change. From rodperson at rodperson.com Sat Jun 24 14:57:21 2017 From: rodperson at rodperson.com (Rod Person) Date: Sat, 24 Jun 2017 14:57:21 -0400 Subject: os.walk the apostrophe and unicode Message-ID: <20170624145721.792c75c8@atomizer-dfly.osb> Hi, I'm working on a program that will walk a file system and clean the id3 tags of mp3 and flac files, everything is working great until the follow file is found '06 - Todd's Song (Post-Spiderland Song in Progress).flac' for some reason that I can't understand os.walk() returns this file name as '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' which then causes more hell than a little bit for me. I'm not understand why apostrophe(') becomes \xe2\x80\x99, or what I can do about it. The script is Python 3, the file system it is running on is a hammer filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which runs some kind of Linux so it probably ext3/4. The files came from various system (Mac, Windows, FreeBSD). -- Rod http://www.rodperson.com From alister.ware at ntlworld.com Sat Jun 24 15:10:38 2017 From: alister.ware at ntlworld.com (alister) Date: Sat, 24 Jun 2017 19:10:38 GMT Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> Message-ID: On Sat, 24 Jun 2017 14:57:21 -0400, Rod Person wrote: > \xe2\x80\x99, because the file name has been created using "Right single quote" instead of apostrophe, the glyphs look identical in many fonts. -- "If you understand what you're doing, you're not learning anything." -- A. L. From john_ladasky at sbcglobal.net Sat Jun 24 15:20:44 2017 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Sat, 24 Jun 2017 12:20:44 -0700 (PDT) Subject: os.walk the apostrophe and unicode In-Reply-To: References: <20170624145721.792c75c8@atomizer-dfly.osb> Message-ID: <96d47496-d149-4c9b-ac8f-ab3f28297a2b@googlegroups.com> On Saturday, June 24, 2017 at 12:07:05 PM UTC-7, Rod Person wrote: > Hi, > > I'm working on a program that will walk a file system and clean the id3 > tags of mp3 and flac files, everything is working great until the > follow file is found > > '06 - Todd's Song (Post-Spiderland Song in Progress).flac' > > for some reason that I can't understand os.walk() returns this file > name as > > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' > > which then causes more hell than a little bit for me. I'm not > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do > about it. That's a "right single quotation mark" character in Unicode. http://unicode.scarfboy.com/?s=E28099 Something in your code is choosing to interpret the text variable as an old-fashioned byte array of characters, where every character is represented by a single byte. That works as long as the file name only uses characters from the old ASCII set, but there are only 128 of those. > The script is Python 3, the file system it is running on is a hammer > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which > runs some kind of Linux so it probably ext3/4. The files came from > various system (Mac, Windows, FreeBSD). Since you are working in Python3, you have the ability to call the .encode() and .decode() methods to translate between Unicode and byte character arrays (which you still need on occasion). > > -- > Rod > > http://www.rodperson.com From python at mrabarnett.plus.com Sat Jun 24 15:25:53 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 24 Jun 2017 20:25:53 +0100 Subject: os.walk the apostrophe and unicode In-Reply-To: <20170624145721.792c75c8@atomizer-dfly.osb> References: <20170624145721.792c75c8@atomizer-dfly.osb> Message-ID: On 2017-06-24 19:57, Rod Person wrote: > Hi, > > I'm working on a program that will walk a file system and clean the id3 > tags of mp3 and flac files, everything is working great until the > follow file is found > > '06 - Todd's Song (Post-Spiderland Song in Progress).flac' > > for some reason that I can't understand os.walk() returns this file > name as > > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' > > which then causes more hell than a little bit for me. I'm not > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do > about it. > > The script is Python 3, the file system it is running on is a hammer > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which > runs some kind of Linux so it probably ext3/4. The files came from > various system (Mac, Windows, FreeBSD). > If you treat it as a bytestring b'\xe2\x80\x99' and decode it: >>> c = b'\xe2\x80\x99'.decode('utf-8') >>> ascii(c) "'\\u2019'" >>> import unicodedata >>> unicodedata.name(c) 'RIGHT SINGLE QUOTATION MARK' It's not an apostrophe, it's '\u2019' ('\N{RIGHT SINGLE QUOTATION MARK}'). It looks like the filename is encoded as UTF-8, but Python thinks that the filesystem encoding is something like Latin-1. From __peter__ at web.de Sat Jun 24 15:28:45 2017 From: __peter__ at web.de (Peter Otten) Date: Sat, 24 Jun 2017 21:28:45 +0200 Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> Message-ID: Rod Person wrote: > Hi, > > I'm working on a program that will walk a file system and clean the id3 > tags of mp3 and flac files, everything is working great until the > follow file is found > > '06 - Todd's Song (Post-Spiderland Song in Progress).flac' > > for some reason that I can't understand os.walk() returns this file > name as > > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' > > which then causes more hell than a little bit for me. I'm not > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do > about it. >>> b"\xe2\x80\x99".decode("utf-8") '?' >>> unicodedata.name(_) 'RIGHT SINGLE QUOTATION MARK' So it's '?' rather than "'". > The script is Python 3, the file system it is running on is a hammer > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which > runs some kind of Linux so it probably ext3/4. The files came from > various system (Mac, Windows, FreeBSD). There seems to be a mismatch between the assumed and the actual file system encoding somewhere in this mix. Is this the only glitch or are there similar problems with other non-ascii characters? From torriem at gmail.com Sat Jun 24 15:28:55 2017 From: torriem at gmail.com (Michael Torrie) Date: Sat, 24 Jun 2017 13:28:55 -0600 Subject: os.walk the apostrophe and unicode In-Reply-To: <20170624145721.792c75c8@atomizer-dfly.osb> References: <20170624145721.792c75c8@atomizer-dfly.osb> Message-ID: <89295c5a-6bae-580f-e03a-83cd08798d11@gmail.com> On 06/24/2017 12:57 PM, Rod Person wrote: > Hi, > > I'm working on a program that will walk a file system and clean the id3 > tags of mp3 and flac files, everything is working great until the > follow file is found > > '06 - Todd's Song (Post-Spiderland Song in Progress).flac' > > for some reason that I can't understand os.walk() returns this file > name as > > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' That's basically a UTF-8 string there: $ python3 >>> a= b'06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' >>> print (a.decode('utf-8')) 06 - Todd?s Song (Post-Spiderland Song in Progress).flac >>> The NAS is just happily reading the UTF-8 bytes and passing them on the wire. > which then causes more hell than a little bit for me. I'm not > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do > about it. It's clearly not an apostrophe in the original filename, but probably U+2019 (?) > The script is Python 3, the file system it is running on is a hammer > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which > runs some kind of Linux so it probably ext3/4. The files came from > various system (Mac, Windows, FreeBSD). It's the file serving protocol that dictates how filenames are transmitted. In your case it's probably smb. smb (samba) is just passing the native bytes along from the file system. Since you know the native file system is just UTF-8, you can just decode every filename from utf-8 bytes into unicode. From rodperson at rodperson.com Sat Jun 24 15:37:52 2017 From: rodperson at rodperson.com (Rod Person) Date: Sat, 24 Jun 2017 15:37:52 -0400 Subject: os.walk the apostrophe and unicode In-Reply-To: References: <20170624145721.792c75c8@atomizer-dfly.osb> Message-ID: <20170624153752.2aaa8e22@atomizer-dfly.osb> On Sat, 24 Jun 2017 21:28:45 +0200 Peter Otten <__peter__ at web.de> wrote: > Rod Person wrote: > > > Hi, > > > > I'm working on a program that will walk a file system and clean the > > id3 tags of mp3 and flac files, everything is working great until > > the follow file is found > > > > '06 - Todd's Song (Post-Spiderland Song in Progress).flac' > > > > for some reason that I can't understand os.walk() returns this file > > name as > > > > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in > > Progress).flac' > > > > which then causes more hell than a little bit for me. I'm not > > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do > > about it. > > >>> b"\xe2\x80\x99".decode("utf-8") > '?' > >>> unicodedata.name(_) > 'RIGHT SINGLE QUOTATION MARK' > > So it's '?' rather than "'". > > > The script is Python 3, the file system it is running on is a hammer > > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS > > which runs some kind of Linux so it probably ext3/4. The files came > > from various system (Mac, Windows, FreeBSD). > > There seems to be a mismatch between the assumed and the actual file > system encoding somewhere in this mix. Is this the only glitch or are > there similar problems with other non-ascii characters? > This is the only glitch as in file names so far. -- Rod http://www.rodperson.com Who at Clitorius fountain thirst remove Loath Wine and, abstinent, meer Water love. - Ovid From rodperson at rodperson.com Sat Jun 24 15:47:25 2017 From: rodperson at rodperson.com (Rod Person) Date: Sat, 24 Jun 2017 15:47:25 -0400 Subject: os.walk the apostrophe and unicode In-Reply-To: <89295c5a-6bae-580f-e03a-83cd08798d11@gmail.com> References: <20170624145721.792c75c8@atomizer-dfly.osb> <89295c5a-6bae-580f-e03a-83cd08798d11@gmail.com> Message-ID: <20170624154725.3c9c76e0@atomizer-dfly.osb> On Sat, 24 Jun 2017 13:28:55 -0600 Michael Torrie wrote: > On 06/24/2017 12:57 PM, Rod Person wrote: > > Hi, > > > > I'm working on a program that will walk a file system and clean the > > id3 tags of mp3 and flac files, everything is working great until > > the follow file is found > > > > '06 - Todd's Song (Post-Spiderland Song in Progress).flac' > > > > for some reason that I can't understand os.walk() returns this file > > name as > > > > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in > > Progress).flac' > > That's basically a UTF-8 string there: > > $ python3 > >>> a= b'06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in > Progress).flac' > >>> print (a.decode('utf-8')) > 06 - Todd?s Song (Post-Spiderland Song in Progress).flac > >>> > > The NAS is just happily reading the UTF-8 bytes and passing them on > the wire. > > > which then causes more hell than a little bit for me. I'm not > > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do > > about it. > > It's clearly not an apostrophe in the original filename, but probably > U+2019 (?) > > > The script is Python 3, the file system it is running on is a hammer > > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS > > which runs some kind of Linux so it probably ext3/4. The files came > > from various system (Mac, Windows, FreeBSD). > > It's the file serving protocol that dictates how filenames are > transmitted. In your case it's probably smb. smb (samba) is just > passing the native bytes along from the file system. Since you know > the native file system is just UTF-8, you can just decode every > filename from utf-8 bytes into unicode. This is the impression that I was under, my unicode is that strong, so maybe my understand is off...but I tried. file_name = file_name.decode('utf-8', 'ignore') but when I get to my logging code: logfile.write(file_name) that throws the error: UnicodeEncodeError: 'ascii' codec can't encode characters in position 39-41: ordinal not in range(128) -- Rod http://www.rodperson.com Who at Clitorius fountain thirst remove Loath Wine and, abstinent, meer Water love. - Ovid From gbs.deadeye at gmail.com Sat Jun 24 16:30:38 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Sat, 24 Jun 2017 20:30:38 +0000 Subject: os.walk the apostrophe and unicode In-Reply-To: <20170624154725.3c9c76e0@atomizer-dfly.osb> References: <20170624145721.792c75c8@atomizer-dfly.osb> <89295c5a-6bae-580f-e03a-83cd08798d11@gmail.com> <20170624154725.3c9c76e0@atomizer-dfly.osb> Message-ID: Can os.fsencode and os.fsdecode help? I've seen it somewhere. I've never used it. To fix encodings, sometimes I use the module ftfy Greetings Andre From python at mrabarnett.plus.com Sat Jun 24 16:34:58 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 24 Jun 2017 21:34:58 +0100 Subject: os.walk the apostrophe and unicode In-Reply-To: <20170624154725.3c9c76e0@atomizer-dfly.osb> References: <20170624145721.792c75c8@atomizer-dfly.osb> <89295c5a-6bae-580f-e03a-83cd08798d11@gmail.com> <20170624154725.3c9c76e0@atomizer-dfly.osb> Message-ID: <5957c750-a972-1826-aee8-7574814e0d8d@mrabarnett.plus.com> On 2017-06-24 20:47, Rod Person wrote: > On Sat, 24 Jun 2017 13:28:55 -0600 > Michael Torrie wrote: > >> On 06/24/2017 12:57 PM, Rod Person wrote: >> > Hi, >> > >> > I'm working on a program that will walk a file system and clean the >> > id3 tags of mp3 and flac files, everything is working great until >> > the follow file is found >> > >> > '06 - Todd's Song (Post-Spiderland Song in Progress).flac' >> > >> > for some reason that I can't understand os.walk() returns this file >> > name as >> > >> > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in >> > Progress).flac' >> >> That's basically a UTF-8 string there: >> >> $ python3 >> >>> a= b'06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in >> Progress).flac' >> >>> print (a.decode('utf-8')) >> 06 - Todd?s Song (Post-Spiderland Song in Progress).flac >> >>> >> >> The NAS is just happily reading the UTF-8 bytes and passing them on >> the wire. >> >> > which then causes more hell than a little bit for me. I'm not >> > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do >> > about it. >> >> It's clearly not an apostrophe in the original filename, but probably >> U+2019 (?) >> >> > The script is Python 3, the file system it is running on is a hammer >> > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS >> > which runs some kind of Linux so it probably ext3/4. The files came >> > from various system (Mac, Windows, FreeBSD). >> >> It's the file serving protocol that dictates how filenames are >> transmitted. In your case it's probably smb. smb (samba) is just >> passing the native bytes along from the file system. Since you know >> the native file system is just UTF-8, you can just decode every >> filename from utf-8 bytes into unicode. > > This is the impression that I was under, my unicode is that strong, so > maybe my understand is off...but I tried. > > file_name = file_name.decode('utf-8', 'ignore') > > but when I get to my logging code: > > logfile.write(file_name) > > that throws the error: > UnicodeEncodeError: 'ascii' codec can't encode characters in > position 39-41: ordinal not in range(128) > > Your logfile was opened with the 'ascii' encoding, so you can't write anything outside the ASCII range. Open it with the 'utf-8' encoding instead. From smith at smith.it Sat Jun 24 16:49:25 2017 From: smith at smith.it (Smith) Date: Sat, 24 Jun 2017 22:49:25 +0200 Subject: environment variable Message-ID: Hello to all, I wanted to ask you how I could delete a line of an environment variable (PATH) ~/Scaricati/pycharm-2017.1.4/bin$ echo $PATH | sed s/:/'\n'/g /usr/local/sbin /usr/local/bin /usr/sbin /usr/bin /sbin /bin /usr/games /usr/local/games /snap/bin /path/to/my/program ---> linea da eliminare /home/dbruno/Scaricati/pycharm-2017.1 Do I unset PATH delete all content? I have not tried yet I downloaded pycharm and wanted to make it executable from the command line without accessing the bin folder and launch the bash script: dbruno at dbruno:~/Scaricati/pycharm-2017.1.4$ ls -la total 48 drwxrwxr-x 10 dbruno dbruno 4096 giu 24 10:23 . drwxr-xr-x 18 dbruno dbruno 4096 giu 24 10:23 .. drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:25 bin -rw-r--r-- 1 dbruno dbruno 14 giu 13 14:48 build.txt drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:23 debug-eggs drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:23 help drwxrwxr-x 17 dbruno dbruno 4096 giu 24 10:23 helpers -rw-r--r-- 1 dbruno dbruno 1887 giu 13 14:48 Install-Linux-tar.txt drwxrwxr-x 4 dbruno dbruno 4096 giu 24 10:23 jre64 drwxrwxr-x 4 dbruno dbruno 4096 giu 24 10:23 lib drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:23 license drwxrwxr-x 52 dbruno dbruno 4096 giu 24 10:23 plugin :~/Scaricati/pycharm-2017.1.4/bin$ ls -la total 7120 drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:25 . drwxrwxr-x 10 dbruno dbruno 4096 giu 24 10:23 .. -rw-rw-r-- 1 dbruno dbruno 0 giu 24 20:18 0M?+ -rwxr-xr-x 1 dbruno dbruno 221 giu 13 14:48 format.sh -rwxr-xr-x 1 dbruno dbruno 23072 giu 13 14:48 fsnotifier -rwxr-xr-x 1 dbruno dbruno 29648 giu 13 14:48 fsnotifier64 -rwxr-xr-x 1 dbruno dbruno 26453 giu 13 14:48 fsnotifier-arm -rw-r--r-- 1 dbruno dbruno 10804 giu 13 14:48 idea.properties -rwxr-xr-x 1 dbruno dbruno 272 giu 13 14:48 inspect.sh -rw-r--r-- 1 dbruno dbruno 3449944 giu 13 14:48 libyjpagent-linux64.so -rw-r--r-- 1 dbruno dbruno 3679036 giu 13 14:48 libyjpagent-linux.so -rw-r--r-- 1 dbruno dbruno 2236 giu 13 14:48 log.xml -rwxr-xr-x 1 dbruno dbruno 410 giu 13 14:48 printenv.py -rw-r--r-- 1 dbruno dbruno 329 giu 13 14:48 pycharm64.vmoptions -rw-r--r-- 1 dbruno dbruno 10281 giu 13 14:48 pycharm.png -rwxr-xr-x 1 dbruno dbruno 6860 giu 13 14:48 pycharm.sh -rw-r--r-- 1 dbruno dbruno 337 giu 13 14:48 pycharm.vmoptions -rwxr-xr-x 1 dbruno dbruno 590 giu 13 14:48 restart.py You can help me ? Thank you From michael.stemper at gmail.com Sat Jun 24 17:14:52 2017 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Sat, 24 Jun 2017 16:14:52 -0500 Subject: environment variable In-Reply-To: References: Message-ID: On 2017-06-24 15:49, Smith wrote: > Hello to all, > I wanted to ask you how I could delete a line of an environment variable > (PATH) > > ~/Scaricati/pycharm-2017.1.4/bin$ echo $PATH | sed s/:/'\n'/g > /usr/local/sbin > /usr/local/bin > /usr/sbin > /usr/bin > /sbin > /bin > /usr/games > /usr/local/games > /snap/bin > /path/to/my/program ---> linea da eliminare > /home/dbruno/Scaricati/pycharm-2017.1 > > Do I unset PATH delete all content? That would not be a good idea. Although this seems to be more of a bash question than a python question, I'll take a crack at it. The first question is: do you want to eliminate that directory from your $PATH forever and ever, or do you just want to eliminate it in some shell instances? If the first is true, find out where it's being added to your $PATH, such as ~/.profile or ~/.bashrc and comment out that line. If the second is true, you could clear it out in real time, with a little something like: . export PATH=`echo $PATH | sed 's,:/path/to/my/program,,'` This has not been tested, but is reasonably close to what's needed. Note that you need the period "." at the beginning to make sure that this happens in the current shell. > I downloaded pycharm and wanted to make it executable from the command > line without accessing the bin folder and launch the bash script: But, here's the big question. How does having "/path/to/my/program" in your $PATH variable prevent you from doing that? The simple answer is that it doesn't. You might want to ADD something to your $PATH, such as "~/Scaricati/pycharm-2017.1.4/bin", but taking something out will not do what you want. Another approach would be to ADD something to your .bash_aliases file, such as: pych() { ~/Scaricati/pycharm-2017.1.4/bin/pycharm.sh $* } Then, you can type "pych" at the command line, and bash will interpret it as the above command, including arguments. Two words of caution here: 1. I haven't tested this code, either. 3. If pycharm does not know its location (which is not necessarily the current directory), it might not be able to find the many accessory files that are there. -- Michael F. Stemper I feel more like I do now than I did when I came in. From __peter__ at web.de Sat Jun 24 17:17:07 2017 From: __peter__ at web.de (Peter Otten) Date: Sat, 24 Jun 2017 23:17:07 +0200 Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> Message-ID: Rod Person wrote: > On Sat, 24 Jun 2017 21:28:45 +0200 > Peter Otten <__peter__ at web.de> wrote: > >> Rod Person wrote: >> >> > Hi, >> > >> > I'm working on a program that will walk a file system and clean the >> > id3 tags of mp3 and flac files, everything is working great until >> > the follow file is found >> > >> > '06 - Todd's Song (Post-Spiderland Song in Progress).flac' >> > >> > for some reason that I can't understand os.walk() returns this file >> > name as >> > >> > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in >> > Progress).flac' >> > >> > which then causes more hell than a little bit for me. I'm not >> > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do >> > about it. >> >> >>> b"\xe2\x80\x99".decode("utf-8") >> '?' >> >>> unicodedata.name(_) >> 'RIGHT SINGLE QUOTATION MARK' >> >> So it's '?' rather than "'". >> >> > The script is Python 3, the file system it is running on is a hammer >> > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS >> > which runs some kind of Linux so it probably ext3/4. The files came >> > from various system (Mac, Windows, FreeBSD). >> >> There seems to be a mismatch between the assumed and the actual file >> system encoding somewhere in this mix. Is this the only glitch or are >> there similar problems with other non-ascii characters? >> > > This is the only glitch as in file names so far. > Then I'd fix the name manually... From gheskett at shentel.net Sat Jun 24 18:39:30 2017 From: gheskett at shentel.net (Gene Heskett) Date: Sat, 24 Jun 2017 18:39:30 -0400 Subject: environment variable In-Reply-To: References: Message-ID: <201706241839.30833.gheskett@shentel.net> On Saturday 24 June 2017 16:49:25 Smith wrote: > Hello to all, > I wanted to ask you how I could delete a line of an environment > variable (PATH) > > ~/Scaricati/pycharm-2017.1.4/bin$ echo $PATH | sed s/:/'\n'/g > /usr/local/sbin > /usr/local/bin > /usr/sbin > /usr/bin > /sbin > /bin > /usr/games > /usr/local/games > /snap/bin > /path/to/my/program ---> linea da eliminare > /home/dbruno/Scaricati/pycharm-2017.1 > > Do I unset PATH delete all content? > I have not tried yet > > I downloaded pycharm and wanted to make it executable from the command > line without accessing the bin folder and launch the bash script: > > dbruno at dbruno:~/Scaricati/pycharm-2017.1.4$ ls -la > total 48 > drwxrwxr-x 10 dbruno dbruno 4096 giu 24 10:23 . > drwxr-xr-x 18 dbruno dbruno 4096 giu 24 10:23 .. > drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:25 bin > -rw-r--r-- 1 dbruno dbruno 14 giu 13 14:48 build.txt > drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:23 debug-eggs > drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:23 help > drwxrwxr-x 17 dbruno dbruno 4096 giu 24 10:23 helpers > -rw-r--r-- 1 dbruno dbruno 1887 giu 13 14:48 Install-Linux-tar.txt > drwxrwxr-x 4 dbruno dbruno 4096 giu 24 10:23 jre64 > drwxrwxr-x 4 dbruno dbruno 4096 giu 24 10:23 lib > drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:23 license > drwxrwxr-x 52 dbruno dbruno 4096 giu 24 10:23 plugin > > :~/Scaricati/pycharm-2017.1.4/bin$ ls -la > > total 7120 > drwxrwxr-x 2 dbruno dbruno 4096 giu 24 10:25 . > drwxrwxr-x 10 dbruno dbruno 4096 giu 24 10:23 .. > -rw-rw-r-- 1 dbruno dbruno 0 giu 24 20:18 0M?+ > -rwxr-xr-x 1 dbruno dbruno 221 giu 13 14:48 format.sh > -rwxr-xr-x 1 dbruno dbruno 23072 giu 13 14:48 fsnotifier > -rwxr-xr-x 1 dbruno dbruno 29648 giu 13 14:48 fsnotifier64 > -rwxr-xr-x 1 dbruno dbruno 26453 giu 13 14:48 fsnotifier-arm > -rw-r--r-- 1 dbruno dbruno 10804 giu 13 14:48 idea.properties > -rwxr-xr-x 1 dbruno dbruno 272 giu 13 14:48 inspect.sh > -rw-r--r-- 1 dbruno dbruno 3449944 giu 13 14:48 > libyjpagent-linux64.so -rw-r--r-- 1 dbruno dbruno 3679036 giu 13 > 14:48 libyjpagent-linux.so -rw-r--r-- 1 dbruno dbruno 2236 giu 13 > 14:48 log.xml > -rwxr-xr-x 1 dbruno dbruno 410 giu 13 14:48 printenv.py > -rw-r--r-- 1 dbruno dbruno 329 giu 13 14:48 pycharm64.vmoptions > -rw-r--r-- 1 dbruno dbruno 10281 giu 13 14:48 pycharm.png > -rwxr-xr-x 1 dbruno dbruno 6860 giu 13 14:48 pycharm.sh > -rw-r--r-- 1 dbruno dbruno 337 giu 13 14:48 pycharm.vmoptions > -rwxr-xr-x 1 dbruno dbruno 590 giu 13 14:48 restart.py > > You can help me ? > > Thank you export PATH= but be prepared to type the full path to anything you want to run. It a PITA, or a cast iron bitch or whatever definition fits. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From cs at zip.com.au Sat Jun 24 19:37:30 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 25 Jun 2017 09:37:30 +1000 Subject: Checking for an exception In-Reply-To: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20170624233730.GA75955@cskk.homeip.net> On 24Jun2017 20:31, Steve D'Aprano wrote: >What's the right/best way to test whether an object is an exception ahead of >time? (That is, without trying to raise from it.) > >I have: > >return (isinstance(obj, type) and issubclass(obj, BaseException) > or isinstance(obj, BaseException)) I haven't a better idea. Are you supporting Python 2 here, where one can raise bare exceptions instead of instances? Also, do you need the "isinstance(obj, type)" precursor to the issubclass? Might you be better with: return ( issubclass(obj, BaseException) if isinstance(obj, type) else isinstance(obj, BaseException) ) ? Curious: why do you need to test this? Some function which may return a "value" or an exception? Cheers, Cameron Simpson From ben+python at benfinney.id.au Sat Jun 24 20:49:20 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 25 Jun 2017 10:49:20 +1000 Subject: Checking for an exception References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85tw34sxgv.fsf@benfinney.id.au> Steve D'Aprano writes: > What's the right/best way to test whether an object is an exception > ahead of time? (That is, without trying to raise from it.) This being Python, it is Easier to Ask for Forgiveness than for Permission. The corollary of that is, if you try to ask permission first (to Look Before You Leap), it will likely not be as easy as simply using the object as you intend to use it. So, EAFP would suggest just raising the object: raise obj and thereby reveal the bug in the *caller's* code if it tries to pass a not-exception object for that purpose. > return (isinstance(obj, type) and issubclass(obj, BaseException) > or isinstance(obj, BaseException)) > > Any better ideas? It's clumsy and noisy, but I think that's the cost of trying to Look Before You Leap in code. -- \ ?My girlfriend has a queen sized bed; I have a court jester | `\ sized bed. It's red and green and has bells on it, and the ends | _o__) curl up.? ?Steven Wright | Ben Finney From steve+python at pearwood.info Sat Jun 24 21:01:05 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 25 Jun 2017 11:01:05 +1000 Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> Message-ID: <594f0b51$0$1614$c3e8da3$5496439d@news.astraweb.com> On Sun, 25 Jun 2017 07:17 am, Peter Otten wrote: > Then I'd fix the name manually... The file name isn't broken. What's broken is parts of the OP's code which assumes that non-ASCII file names are broken... -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Jun 24 21:03:18 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 25 Jun 2017 11:03:18 +1000 Subject: environment variable References: <201706241839.30833.gheskett@shentel.net> Message-ID: <594f0bd7$0$1583$c3e8da3$5496439d@news.astraweb.com> On Sun, 25 Jun 2017 08:39 am, Gene Heskett wrote: > On Saturday 24 June 2017 16:49:25 Smith wrote: > >> Hello to all, >> I wanted to ask you how I could delete a line of an environment >> variable (PATH) [...] > export PATH= > > but be prepared to type the full path to anything you want to run. It a > PITA, or a cast iron bitch or whatever definition fits. Are you serious? The OP asked how to delete *one line* from an environment variable, and you respond by deleting them all? "The ashtray in my car is full, how do I clean it?" "Throw the car away and walk everywhere." -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Jun 24 21:15:01 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 25 Jun 2017 11:15:01 +1000 Subject: Checking for an exception References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> <20170624233730.GA75955@cskk.homeip.net> Message-ID: <594f0e96$0$1608$c3e8da3$5496439d@news.astraweb.com> On Sun, 25 Jun 2017 09:37 am, Cameron Simpson wrote: > On 24Jun2017 20:31, Steve D'Aprano wrote: >>What's the right/best way to test whether an object is an exception ahead of >>time? (That is, without trying to raise from it.) >> >>I have: >> >>return (isinstance(obj, type) and issubclass(obj, BaseException) >> or isinstance(obj, BaseException)) > > I haven't a better idea. > > Are you supporting Python 2 here, where one can raise bare exceptions instead > of instances? Both Python 2 and Python 3 support raising either exception classes or exception instances: raise ValueError raise ValueError('message') both work. (If you use a type alone, the raise statement automatically instantiates it, otherwise it just uses the instance you give.) > Also, do you need the "isinstance(obj, type)" precursor to the > issubclass? Yes, because annoyingly issubclass raises if you pass something which isn't a type, instead of just returning False: py> issubclass(99, int) Traceback (most recent call last): File "", line 1, in TypeError: issubclass() arg 1 must be a class > Might you be better with: > > return ( issubclass(obj, BaseException) > if isinstance(obj, type) > else isinstance(obj, BaseException) > ) > > ? I didn't think of that. If I were just supporting Python 2.7 or 3.6, I think I would prefer that, but for my sins I'm supporting 2.4 :-( > Curious: why do you need to test this? Some function which may return a > "value" or an exception? I have a decorator which converts exceptions in the decorated function from one type to another. It requires as two arguments: - the exception to be caught: an exception type, or a tuple of exception types - the exception to be re-raised: an exception type, or an exception instance If the caller provides bad arguments to the decorator, I want to raise *immediately*, not when the decorated function is called. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From gheskett at shentel.net Sat Jun 24 21:31:00 2017 From: gheskett at shentel.net (Gene Heskett) Date: Sat, 24 Jun 2017 21:31:00 -0400 Subject: environment variable In-Reply-To: <594f0bd7$0$1583$c3e8da3$5496439d@news.astraweb.com> References: <594f0bd7$0$1583$c3e8da3$5496439d@news.astraweb.com> Message-ID: <201706242131.00877.gheskett@shentel.net> On Saturday 24 June 2017 21:03:18 Steve D'Aprano wrote: > On Sun, 25 Jun 2017 08:39 am, Gene Heskett wrote: > > On Saturday 24 June 2017 16:49:25 Smith wrote: > >> Hello to all, > >> I wanted to ask you how I could delete a line of an environment > >> variable (PATH) > > [...] > > > export PATH= > > > > but be prepared to type the full path to anything you want to run. > > It a PITA, or a cast iron bitch or whatever definition fits. > > Are you serious? The OP asked how to delete *one line* from an > environment variable, and you respond by deleting them all? > $PATH is a single line in the env. The OP asked how to delete it, so I told him. If he had asked how to get rid of a single piece of the $PATH, then my answer would have been longer and more precise. > "The ashtray in my car is full, how do I clean it?" Take it out and dump it. Put it back in. Or leave it out, the car will not care, it will still haul your butt as long as theres oil in the engine and gas in the tank. > "Throw the car away and walk everywhere." This is one of those posts that I should have ignored as this list has an abundance of standup comics anyway. > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and > sure enough, things got worse. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From steve+python at pearwood.info Sat Jun 24 21:31:27 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 25 Jun 2017 11:31:27 +1000 Subject: Checking for an exception References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> <85tw34sxgv.fsf@benfinney.id.au> Message-ID: <594f1270$0$1597$c3e8da3$5496439d@news.astraweb.com> On Sun, 25 Jun 2017 10:49 am, Ben Finney wrote: > Steve D'Aprano writes: > >> What's the right/best way to test whether an object is an exception >> ahead of time? (That is, without trying to raise from it.) > > This being Python, it is Easier to Ask for Forgiveness than for > Permission. Sometimes... > The corollary of that is, if you try to ask permission first (to Look > Before You Leap), it will likely not be as easy as simply using the > object as you intend to use it. > > So, EAFP would suggest just raising the object: > > raise obj Unfortunately it's not that simple, as the result of passing a non-exception to raise is to raise an exception, so I cannot trivially distinguish between "caller passes an exception" and "caller passes a non-exception" (result is still an exception). I could write something like: def is_exception(obj): try: raise obj # This unconditionally raises. except TypeError: # Either obj is an instance or subclass of TypeError, # or it's some arbitrary non-exception object. if isinstance(obj, TypeError): return True try: return issubclass(obj, TypeError) except TypeError: return False except: # Any exception other than TypeError means obj is that exception. return True else: # In Python 2.4 and 2.5 you can (but shouldn't) raise strings. assert isinstance(obj, str) return False but I don't think that's much of an improvement :-) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben+python at benfinney.id.au Sat Jun 24 23:47:50 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 25 Jun 2017 13:47:50 +1000 Subject: Checking for an exception References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> <85tw34sxgv.fsf@benfinney.id.au> <594f1270$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85podssp7d.fsf@benfinney.id.au> Steve D'Aprano writes: > [?] the result of passing a non-exception to raise is to raise an > exception, so I cannot trivially distinguish between "caller passes an > exception" and "caller passes a non-exception" (result is still an > exception). Yes, hence my characterising this problem as the caller's problem. I'd say: document the expectation that the value will be an exception, use it based on that specification, and let the caller deal with the consequences of violating that expectation. -- \ ?I went to court for a parking ticket; I pleaded insanity. I | `\ said ?Your Honour, who in their right mind parks in the passing | _o__) lane??? ?Steven Wright | Ben Finney From cs at zip.com.au Sun Jun 25 00:07:38 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 25 Jun 2017 14:07:38 +1000 Subject: Checking for an exception In-Reply-To: <85podssp7d.fsf@benfinney.id.au> References: <85podssp7d.fsf@benfinney.id.au> Message-ID: <20170625040738.GA79511@cskk.homeip.net> On 25Jun2017 13:47, Ben Finney wrote: >Steve D'Aprano writes: >> [?] the result of passing a non-exception to raise is to raise an >> exception, so I cannot trivially distinguish between "caller passes an >> exception" and "caller passes a non-exception" (result is still an >> exception). > >Yes, hence my characterising this problem as the caller's problem. > >I'd say: document the expectation that the value will be an exception, >use it based on that specification, and let the caller deal with the >consequences of violating that expectation. I'm a "fail early" kind of guy, and to me Steve's approach is in the same spirit as raising ValueError when a function is handed invalid arguments. Particularly if the mistake is easy to make, having one's attention brought to it immediately (at "declaration" time, since Steve's example is a decorator), seems very desirable. Cheers, Cameron Simpson From cs at zip.com.au Sun Jun 25 00:47:40 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 25 Jun 2017 14:47:40 +1000 Subject: environment variable In-Reply-To: <201706242131.00877.gheskett@shentel.net> References: <201706242131.00877.gheskett@shentel.net> Message-ID: <20170625044740.GA27935@cskk.homeip.net> On 24Jun2017 21:31, Gene Heskett wrote: >On Saturday 24 June 2017 21:03:18 Steve D'Aprano wrote: >> On Sun, 25 Jun 2017 08:39 am, Gene Heskett wrote: >> > On Saturday 24 June 2017 16:49:25 Smith wrote: >> >> Hello to all, >> >> I wanted to ask you how I could delete a line of an environment >> >> variable (PATH) >> >> [...] >> > export PATH= >> > but be prepared to type the full path to anything you want to run. >> > It a PITA, or a cast iron bitch or whatever definition fits. >> >> Are you serious? The OP asked how to delete *one line* from an >> environment variable, and you respond by deleting them all? >> >$PATH is a single line in the env. The OP asked how to delete it, so I >told him. If he had asked how to get rid of a single piece of the $PATH, >then my answer would have been longer and more precise. English is clearly the OP's second language, and his post was pretty clear that he did indeed want to remove a single item from the things named in $PATH. He even provided example code showing exactly what he wanted to remove, and which demonstrated that he knew it was a colon separated string, which he was presenting as lines for purposes of clarity. Cheers, Cameron Simpson From __peter__ at web.de Sun Jun 25 02:57:47 2017 From: __peter__ at web.de (Peter Otten) Date: Sun, 25 Jun 2017 08:57:47 +0200 Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> <594f0b51$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > On Sun, 25 Jun 2017 07:17 am, Peter Otten wrote: > >> Then I'd fix the name manually... > > The file name isn't broken. > > > What's broken is parts of the OP's code which assumes that non-ASCII file > names are broken... Hm, the OP says '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' Shouldn't it be '06 - Todd?s Song (Post-Spiderland Song in Progress).flac' if everything worked correctly? Though I don't understand why the OP doesn't see '06 - Todd?\x80\x99s Song (Post-Spiderland Song in Progress).flac' which is the repr() that I get. From no.email at nospam.invalid Sun Jun 25 03:50:50 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 25 Jun 2017 00:50:50 -0700 Subject: Checking for an exception References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87tw34zesl.fsf@nightsong.com> Steve D'Aprano writes: > What's the right/best way to test whether an object is an exception > ahead of time? (That is, without trying to raise from it.) Maybe I'm missing something but isinstance(obj, Exception) seems to work. From steve+python at pearwood.info Sun Jun 25 03:53:31 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 25 Jun 2017 17:53:31 +1000 Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> <594f0b51$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: <594f6bfc$0$1614$c3e8da3$5496439d@news.astraweb.com> On Sun, 25 Jun 2017 04:57 pm, Peter Otten wrote: > Steve D'Aprano wrote: > >> On Sun, 25 Jun 2017 07:17 am, Peter Otten wrote: >> >>> Then I'd fix the name manually... >> >> The file name isn't broken. >> >> >> What's broken is parts of the OP's code which assumes that non-ASCII file >> names are broken... > > Hm, the OP says > > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' > > Shouldn't it be > > '06 - Todd?s Song (Post-Spiderland Song in Progress).flac' It should, if the OP did everything right. He has a file name containing the word "Todd?s": # Python 3.5 py> fname = 'Todd?s' py> repr(fname) "'Todd?s'" On disk, that is represented in UTF-8: py> repr(fname.encode('utf-8')) "b'Todd\\xe2\\x80\\x99s'" The OP appears to be using Python 2, so when he calls os.listdir() he gets the file names as bytes, not Unicode. That means he'll see: - the file name will be Python 2 str, which is *byte string* not text string; - so not Unicode - rather the individual bytes in the UTF-8 encoding of the file name. So in Python 2.7 instead of 3.5 above: py> fname = u'Todd?s' py> repr(fname) "u'Todd\\u2019s'" py> repr(fname.encode('utf-8')) "'Todd\\xe2\\x80\\x99s'" > if everything worked correctly? Though I don't understand why the OP doesn't > see > > '06 - Todd?\x80\x99s Song (Post-Spiderland Song in Progress).flac' > > which is the repr() that I get. That's mojibake and is always wrong :-) I'm not sure how you got that. Something to do with an accidental decode to Latin-1? # Python 2.7 py> repr(fname.encode('utf-8').decode('latin-1')) "u'Todd\\xe2\\x80\\x99s'" # Python 3.5 py> repr(fname.encode('utf-8').decode('latin-1')) "'Todd?\\x80\\x99s'" -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From __peter__ at web.de Sun Jun 25 04:47:18 2017 From: __peter__ at web.de (Peter Otten) Date: Sun, 25 Jun 2017 10:47:18 +0200 Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> <594f0b51$0$1614$c3e8da3$5496439d@news.astraweb.com> <594f6bfc$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > On Sun, 25 Jun 2017 04:57 pm, Peter Otten wrote: >> if everything worked correctly? Though I don't understand why the OP >> doesn't see >> >> '06 - Todd?\x80\x99s Song (Post-Spiderland Song in Progress).flac' >> >> which is the repr() that I get. > > That's mojibake and is always wrong :-) Yes, that's my very point. > I'm not sure how you got that. I took the OP's string at face value and pasted it into the interpreter: # python 3.4 >>> '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac' '06 - Todd?\x80\x99s Song (Post-Spiderland Song in Progress).flac' > Something to do with an accidental decode to Latin-1? If the above filename is the only one or one of a few that seem broken, and other non-ascii filenames look OK the OP's toolchain/filesystem may work correctly and the odd name might have been produced elsewhere, e. g. by copying an already messed-up freedb.org entry. [Heureka] However, the most likely explanation is that the filename is correct and that the OP is not using Python 3 as he claims but Python 2. Yes, it took that long for me to realise ;) Python 2 is slowly sinking into oblivion... From alister.ware at ntlworld.com Sun Jun 25 07:38:56 2017 From: alister.ware at ntlworld.com (alister) Date: Sun, 25 Jun 2017 11:38:56 GMT Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> Message-ID: On Sun, 25 Jun 2017 02:23:15 -0700, wxjmfauth wrote: > Le samedi 24 juin 2017 21:10:47 UTC+2, alister a ?crit?: >> On Sat, 24 Jun 2017 14:57:21 -0400, Rod Person wrote: >> >> > \xe2\x80\x99, >> >> because the file name has been created using "Right single quote" >> instead of apostrophe, the glyphs look identical in many fonts. >> >> > Trust me. Fonts are clearly making distinction between \u0027 and > \u2019. Not all, and even when they do it has absolutely nothing to do with the point of the post the character in the file name is \u2019 right quotation mark & not an apostrophe which the op was assuming. he needs to decode the file name correctly -- You will be held hostage by a radical group. From rodperson at rodperson.com Sun Jun 25 08:19:02 2017 From: rodperson at rodperson.com (Rod Person) Date: Sun, 25 Jun 2017 08:19:02 -0400 Subject: os.walk the apostrophe and unicode In-Reply-To: References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> <594f0b51$0$1614$c3e8da3$5496439d@news.astraweb.com> <594f6bfc$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20170625081902.6973a93b@atomizer-dfly.osb> Ok...so after reading all the replies in the thread, I thought I would be easier to send a general reply and include some links to screenshots. As Peter mention, the logic thing to do would be to fix the file name to what I actually thought it was and if this was for work that probably what I would have done, but since I want to understand what's going on I decided to waste time on that. I have to admit, I didn't think the file system was utf-8 as seeing what looked to be an apostrophe sent me down the road of why is this apostrophe screwed up instead of "ah this must be unicode". But doing a simple ls of that directory show it is unicode but the replacement of the offending character. http://rodperson.com/graphics/uc/ls.png I am in fact using Python 3.5. I may be lacking in unicode skills but I do have the sense enough to know the version of Python I am invoking. So I included this screenshot of that so the version of Python and the files list returned by os.walk http://rodperson.com/graphics/uc/files.png So the fact that it shows as a string and not bytes in the debugger was throwing me for a loop, in my log section I was trying to determine if it was unicode decode it...if not don't do anything which wasn't working http://rodperson.com/graphics/uc/log_section.png On Sun, 25 Jun 2017 10:47:18 +0200 Peter Otten <__peter__ at web.de> wrote: > Steve D'Aprano wrote: > > > On Sun, 25 Jun 2017 04:57 pm, Peter Otten wrote: > > >> if everything worked correctly? Though I don't understand why the > >> OP doesn't see > >> > >> '06 - Todd?\x80\x99s Song (Post-Spiderland Song in Progress).flac' > >> > >> which is the repr() that I get. > > > > That's mojibake and is always wrong :-) > > Yes, that's my very point. > > > I'm not sure how you got that. > > I took the OP's string at face value and pasted it into the > interpreter: > > # python 3.4 > >>> '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in > >>> Progress).flac' > '06 - Todd?\x80\x99s Song (Post-Spiderland Song in Progress).flac' > > > Something to do with an accidental decode to Latin-1? > > If the above filename is the only one or one of a few that seem > broken, and other non-ascii filenames look OK the OP's > toolchain/filesystem may work correctly and the odd name might have > been produced elsewhere, e. g. by copying an already messed-up > freedb.org entry. > > [Heureka] > > However, the most likely explanation is that the filename is correct > and that the OP is not using Python 3 as he claims but Python 2. > > Yes, it took that long for me to realise ;) Python 2 is slowly > sinking into oblivion... > -- Rod http://www.rodperson.com From torriem at gmail.com Sun Jun 25 10:18:45 2017 From: torriem at gmail.com (Michael Torrie) Date: Sun, 25 Jun 2017 08:18:45 -0600 Subject: os.walk the apostrophe and unicode In-Reply-To: <20170625081902.6973a93b@atomizer-dfly.osb> References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> <594f0b51$0$1614$c3e8da3$5496439d@news.astraweb.com> <594f6bfc$0$1614$c3e8da3$5496439d@news.astraweb.com> <20170625081902.6973a93b@atomizer-dfly.osb> Message-ID: <8758ad6b-83a7-efe1-d2a3-cbdfb330887b@gmail.com> On 06/25/2017 06:19 AM, Rod Person wrote: > But doing a simple ls of that directory show it is unicode but the > replacement of the offending character. > > http://rodperson.com/graphics/uc/ls.png Now that is really strange. Your OS seems to not recognize that the filename is in UTF-8. I suspect this has something to do with the NAS file sharing protocol (smb). Though I'm pretty sure that Samba can handle UTF-8 filenames correctly. > I am in fact using Python 3.5. I may be lacking in unicode skills but I > do have the sense enough to know the version of Python I am invoking. > So I included this screenshot of that so the version of Python and the > files list returned by os.walk > > http://rodperson.com/graphics/uc/files.png If I create a file that has the U+2019 character in it on my Linux machine (BtrFS), and do os.walk on it, I see the character in then string properly. So it looks like Python does the right thing, automatically decoding from UTF-8. In your situation I think the problem is the file sharing protocol that your NAS is using. Somehow some information is being lost and your OS does not know that the filenames are in UTF-8, and just thinks they are bytes. And therefore Python doesn't know to decode the string, so you just end up with each byte being converted to a unicode code point and being shoved into the unicode string. How to get around this issue I don't know. Maybe there's a way to convert the unicode string to bytes using the value of each character, and then decode that back to unicode. From __peter__ at web.de Sun Jun 25 10:28:17 2017 From: __peter__ at web.de (Peter Otten) Date: Sun, 25 Jun 2017 16:28:17 +0200 Subject: os.walk the apostrophe and unicode References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> <594f0b51$0$1614$c3e8da3$5496439d@news.astraweb.com> <594f6bfc$0$1614$c3e8da3$5496439d@news.astraweb.com> <20170625081902.6973a93b@atomizer-dfly.osb> Message-ID: Rod Person wrote: > Ok...so after reading all the replies in the thread, I thought I would > be easier to send a general reply and include some links to screenshots. > > As Peter mention, the logic thing to do would be to fix the file name > to what I actually thought it was and if this was for work that > probably what I would have done, but since I want to understand what's > going on I decided to waste time on that. > > I have to admit, I didn't think the file system was utf-8 as seeing what > looked to be an apostrophe sent me down the road of why is this > apostrophe screwed up instead of "ah this must be unicode". > > But doing a simple ls of that directory show it is unicode but the > replacement of the offending character. > > http://rodperson.com/graphics/uc/ls.png Have you set LANG to something that implies ASCII? $ touch Todd?s ?hnlich ?blich l?blich $ ls ?hnlich l?blich Todd?s ?blich $ LANG=C ls Todd???s l??blich ??hnlich ??blich $ python3 -c 'import os; print(os.listdir())' ['Todd?s', '?blich', '?hnlich', 'l?blich'] $ LANG=C python3 -c 'import os; print(os.listdir())' ['Todd\udce2\udc80\udc99s', '\udcc3\udcbcblich', '\udcc3\udca4hnlich', 'l\udcc3\udcb6blich'] $ LANG=en_US.utf-8 python3 -c 'import os; print(os.listdir())' ['Todd?s', '?blich', '?hnlich', 'l?blich'] For file names Python resorts to surrogates whenever a byte does not translate into a character in the advertised encoding. > I am in fact using Python 3.5. I may be lacking in unicode skills but I > do have the sense enough to know the version of Python I am invoking. I've made so many "stupid errors" myself that I always consider them first ;) > So I included this screenshot of that so the version of Python and the > files list returned by os.walk > > http://rodperson.com/graphics/uc/files.png > > So the fact that it shows as a string and not bytes in the debugger was > throwing me for a loop, in my log section I was trying to determine if > it was unicode decode it...if not don't do anything which wasn't working > > http://rodperson.com/graphics/uc/log_section.png > > > > > On Sun, 25 Jun 2017 10:47:18 +0200 > Peter Otten <__peter__ at web.de> wrote: > >> Steve D'Aprano wrote: >> >> > On Sun, 25 Jun 2017 04:57 pm, Peter Otten wrote: >> >> >> if everything worked correctly? Though I don't understand why the >> >> OP doesn't see >> >> >> >> '06 - Todd?\x80\x99s Song (Post-Spiderland Song in Progress).flac' >> >> >> >> which is the repr() that I get. >> > >> > That's mojibake and is always wrong :-) >> >> Yes, that's my very point. >> >> > I'm not sure how you got that. >> >> I took the OP's string at face value and pasted it into the >> interpreter: >> >> # python 3.4 >> >>> '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in >> >>> Progress).flac' >> '06 - Todd?\x80\x99s Song (Post-Spiderland Song in Progress).flac' >> >> > Something to do with an accidental decode to Latin-1? >> >> If the above filename is the only one or one of a few that seem >> broken, and other non-ascii filenames look OK the OP's >> toolchain/filesystem may work correctly and the odd name might have >> been produced elsewhere, e. g. by copying an already messed-up >> freedb.org entry. >> >> [Heureka] >> >> However, the most likely explanation is that the filename is correct >> and that the OP is not using Python 3 as he claims but Python 2. >> >> Yes, it took that long for me to realise ;) Python 2 is slowly >> sinking into oblivion... >> > > > From rodperson at rodperson.com Sun Jun 25 11:14:53 2017 From: rodperson at rodperson.com (Rod Person) Date: Sun, 25 Jun 2017 11:14:53 -0400 Subject: os.walk the apostrophe and unicode In-Reply-To: <8758ad6b-83a7-efe1-d2a3-cbdfb330887b@gmail.com> References: <20170624145721.792c75c8@atomizer-dfly.osb> <20170624153752.2aaa8e22@atomizer-dfly.osb> <594f0b51$0$1614$c3e8da3$5496439d@news.astraweb.com> <594f6bfc$0$1614$c3e8da3$5496439d@news.astraweb.com> <20170625081902.6973a93b@atomizer-dfly.osb> <8758ad6b-83a7-efe1-d2a3-cbdfb330887b@gmail.com> Message-ID: <20170625111453.3fcb79d0@atomizer-dfly.osb> On Sun, 25 Jun 2017 08:18:45 -0600 Michael Torrie wrote: > On 06/25/2017 06:19 AM, Rod Person wrote: > > But doing a simple ls of that directory show it is unicode but the > > replacement of the offending character. > > > > http://rodperson.com/graphics/uc/ls.png > > Now that is really strange. Your OS seems to not recognize that the > filename is in UTF-8. I suspect this has something to do with the NAS > file sharing protocol (smb). Though I'm pretty sure that Samba can > handle UTF-8 filenames correctly. > > > I am in fact using Python 3.5. I may be lacking in unicode skills > > but I do have the sense enough to know the version of Python I am > > invoking. So I included this screenshot of that so the version of > > Python and the files list returned by os.walk > > > > http://rodperson.com/graphics/uc/files.png > > If I create a file that has the U+2019 character in it on my Linux > machine (BtrFS), and do os.walk on it, I see the character in then > string properly. So it looks like Python does the right thing, > automatically decoding from UTF-8. > > In your situation I think the problem is the file sharing protocol > that your NAS is using. Somehow some information is being lost and > your OS does not know that the filenames are in UTF-8, and just > thinks they are bytes. And therefore Python doesn't know to decode > the string, so you just end up with each byte being converted to a > unicode code point and being shoved into the unicode string. > > How to get around this issue I don't know. Maybe there's a way to > convert the unicode string to bytes using the value of each character, > and then decode that back to unicode. I think you theory is on the correct path. I'm actually attached to the NAS via NFS not samba. And just quickly looking into that it seems the NFS server needs and option set to pass unicode correctly...but my NAS software doesn't allow my access to settings only to turn it on or off. Looks like my option is the original correct the file name. -- Rod http://www.rodperson.com Who at Clitorius fountain thirst remove Loath Wine and, abstinent, meer Water love. - Ovid From steve+python at pearwood.info Sun Jun 25 12:10:03 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 26 Jun 2017 02:10:03 +1000 Subject: Checking for an exception References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> <87tw34zesl.fsf@nightsong.com> Message-ID: <594fe05d$0$1591$c3e8da3$5496439d@news.astraweb.com> On Sun, 25 Jun 2017 05:50 pm, Paul Rubin wrote: > Steve D'Aprano writes: >> What's the right/best way to test whether an object is an exception >> ahead of time? (That is, without trying to raise from it.) > > Maybe I'm missing something but > isinstance(obj, Exception) > seems to work. Not well enough I'm afraid: py> isinstance(KeyboardInterrupt(), Exception) False py> isinstance(ValueError, Exception) False -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From skip.montanaro at gmail.com Sun Jun 25 12:40:28 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 25 Jun 2017 11:40:28 -0500 Subject: Checking for an exception In-Reply-To: <594fe05d$0$1591$c3e8da3$5496439d@news.astraweb.com> References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> <87tw34zesl.fsf@nightsong.com> <594fe05d$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: > > py> isinstance(KeyboardInterrupt(), Exception) > False > py> isinstance(ValueError, Exception) > False > I might have missed something, but don't you want to be using BaseException as your class/type? Also, Checking isinstance() between two classes isn't likely to work, I don't think. Both the 2.7 and 3.6 docs indicate that BaseException is the base class for all built-in exceptions: https://docs.python.org/2/library/exceptions.html https://docs.python.org/3/library/exceptions.html Of course, YMMV when dealing with user-defined exception classes. Skip From darcy at VybeNetworks.com Sun Jun 25 12:48:43 2017 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Sun, 25 Jun 2017 12:48:43 -0400 Subject: Checking for an exception In-Reply-To: <594fe05d$0$1591$c3e8da3$5496439d@news.astraweb.com> References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> <87tw34zesl.fsf@nightsong.com> <594fe05d$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: <94c05d51-1416-8093-d879-a2e4f06bb6ad@VybeNetworks.com> On 06/25/17 12:10, Steve D'Aprano wrote: > py> isinstance(KeyboardInterrupt(), Exception) > False > py> isinstance(ValueError, Exception) > False That's because KeyboardInterrupt is not a subclass of Exception. If you want to catch that as well you need to check against BaseException. https://docs.python.org/3.6/library/exceptions.html#exception-hierarchy -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From steve+python at pearwood.info Sun Jun 25 13:26:08 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 26 Jun 2017 03:26:08 +1000 Subject: exception_guard context manager and decorator Message-ID: <594ff232$0$1610$c3e8da3$5496439d@news.astraweb.com> As discussed in the Python-Ideas mailing list, sometimes we want to suppress a particular kind of exception and replace it with another. For that reason, I'd like to announce exception_guard, a context manager and decorator which catches specified exceptions and replaces them with a given exception (RuntimeError by default). https://code.activestate.com/recipes/580808-guard-against-an-exception-in-the-wrong-place/ or just https://code.activestate.com/recipes/580808 It should work with Python 2.6 through 3.6 and later. try: with exception_guard(ZeroDivisionError): 1/0 # raises ZeroDivisionError except RuntimeError: print ('ZeroDivisionError replaced by RuntimeError') See the recipe on ActiveState for links to discussions on Python-Ideas. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Jun 25 13:31:06 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 26 Jun 2017 03:31:06 +1000 Subject: Checking for an exception References: <594e3f67$0$1622$c3e8da3$5496439d@news.astraweb.com> <87tw34zesl.fsf@nightsong.com> <594fe05d$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: <594ff35d$0$1583$c3e8da3$5496439d@news.astraweb.com> On Mon, 26 Jun 2017 02:40 am, Skip Montanaro wrote: >> py> isinstance(KeyboardInterrupt(), Exception) >> False >> py> isinstance(ValueError, Exception) >> False >> > > I might have missed something, but don't you want to be using BaseException > as your class/type? Yes I do, which is why I was demonstrating to Paul that his suggestion to use Exception instead of BaseException is not good enough. *wink* > Also, Checking isinstance() between two classes isn't > likely to work, I don't think. Indeed it doesn't, which is why I was demonstrating that Paul's suggestion to use isinstance instead of issubclass is not good enough. *wink* > Both the 2.7 and 3.6 docs indicate that BaseException is the base class for > all built-in exceptions: Indeed it is. Which is why in my original post I used BaseException. *eye spasms from winking too much* > https://docs.python.org/2/library/exceptions.html > https://docs.python.org/3/library/exceptions.html > > Of course, YMMV when dealing with user-defined exception classes. No, user-defined exceptions must derive from BaseException too. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From miragewebstudio12 at gmail.com Sun Jun 25 14:29:14 2017 From: miragewebstudio12 at gmail.com (Mirage Web Studio) Date: Sun, 25 Jun 2017 23:59:14 +0530 Subject: comp.lang.python killfile rule In-Reply-To: References: Message-ID: Just felt like posting, wouldn't it be pythonic if it was if word in [list]: ignore Save time and easily maintainable Cmg On 23 Jun 2017 02:41, "John Black" wrote: All, in case this is useful to anyone, this rule that tells my newsreader which posts to kill really cleans up the group. I could not find a way to key off of anything in the header except keywords because the From keeps changing and I didn't want to overkill real posts. I may have to add a thing or two to this over time, but right now, this seems to be nailing everything. John Black Subject contains "PEDOFILO" Or Subject contains "MAI" Or Subject contains "SEGRETO" Or Subject contains "SETTA" Or Subject contains "BAMBINI" Or Subject contains "FIGLIO" Or Subject contains "PAOLO" Or Subject contains "NATALE" Or Subject contains "SONO" Or Subject contains "GRAZIA" Or Subject contains "PORNOSTAR" Or Subject contains "PEZZO" Or Subject contains "MERDA" Or Subject contains "CAZZO" Or Subject contains "GALERA" Or Subject contains "SICARIO" Or Subject contains "ESSERE" Or Subject contains "CRIMINALE" Or Subject contains "LECCA" Or Subject contains "COCAINA" Or Subject contains "LESBICA" Or Subject contains "NESSUNO" Or Subject contains "MAFIOSO" Or Subject contains "BERLUSCONI" Or Subject contains "????" Or Subject contains "HARDCORE" Or Subject contains "PEDERASTA" Or Subject contains "CULO" Or Subject contains "NOSTRA" Or Subject contains "FOGLIO" Or Subject contains "USARE" Or Subject contains "FAMIGLIA" Or Subject contains "FECE" Or Subject contains "CAPO" Or Subject contains "SUICIDARE" Or Subject contains "OGNI" Or Subject contains "CANE" Or Subject contains "MERCATO" Or Subject contains "VOLTA" Or Subject contains "MAFIOSA" Or Subject contains "ALMENO" Or Subject contains "BASTARDO" Or Subject contains "FIGLIA" Or Subject contains "BASTARD" Or Subject contains "CRIMINAL" Or Subject contains "ANNI" Or Subject contains "PEDINA" -- https://mail.python.org/mailman/listinfo/python-list From steve+python at pearwood.info Sun Jun 25 22:04:04 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 26 Jun 2017 12:04:04 +1000 Subject: comments and the continuation prompt References: Message-ID: <59506b95$0$1611$c3e8da3$5496439d@news.astraweb.com> On Mon, 26 Jun 2017 08:44 am, Stefan Ram wrote: > When I enter ?12\?, I get a continuation prompt in the > Python 3.6 console: > >>>> 12\ > ... > > . I thought that this might indicate that the logical line > is not terminated yet. No. You get the level 2 prompt (sys.ps2) for a number of reasons: - after statements that require a block (def, class, if, while, for, with, try, and any others I have missed); - in triple-quoted strings; - bracketed expressions which haven't been closed yet ( [ { - after comments. > According to The Python Language Reference Release 3.6.0, > 2.1.3 Comments, ?A comment signifies the end of the logical > line unless the implicit line joining rules are invoked.?. > > So, why do I get a continuation prompt when I enter a comment? Why not? As far as the interactive interpreter is concerned, you haven't yet entered a statement. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben+python at benfinney.id.au Sun Jun 25 23:32:42 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 26 Jun 2017 13:32:42 +1000 Subject: comments and the continuation prompt References: <59506b95$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <85k23zs9t1.fsf@benfinney.id.au> Steve D'Aprano writes: > On Mon, 26 Jun 2017 08:44 am, Stefan Ram wrote: > > > According to The Python Language Reference Release 3.6.0, 2.1.3 > > Comments, ?A comment signifies the end of the logical line unless > > the implicit line joining rules are invoked.?. > > > > So, why do I get a continuation prompt when I enter a comment? > > Why not? As far as the interactive interpreter is concerned, you > haven't yet entered a statement. And yet, according to the Language Reference, the logical line has ended and another begun. So I think the question is worth exploring: Why does the interactive prompt imply the logical line is continuing, when the Language Reference would say otherwise? Maybe the answer is ?the continuation prompt does not prompt for the continuation of a logical line, but the continuation of ?. What exactly goes in the ?? placeholder; that is, exactly what should the user understand by that transition from one prompt to a different one? -- \ ?Philosophy is questions that may never be answered. Religion | `\ is answers that may never be questioned.? ?anonymous | _o__) | Ben Finney From quintin9g at gmail.com Mon Jun 26 00:08:34 2017 From: quintin9g at gmail.com (Edward Montague) Date: Mon, 26 Jun 2017 16:08:34 +1200 Subject: GUI Designer[s] Message-ID: I've become a bit more familiar with wxglade , wxFormBuilder and to a lesser extent BoaConstructor. There are however numerous other possiblities and extensions . I'd like to eventually have 3D graphics within an application constructed through a GUI Designer ; preferably with quality approaching or exceeding that of MayaVI2. Is there any project underway to meld the plethora of possibilties into something more comprehensive as a GUI Designer. From ben+python at benfinney.id.au Mon Jun 26 00:29:50 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 26 Jun 2017 14:29:50 +1000 Subject: GUI Designer[s] References: Message-ID: <85fuens75t.fsf@benfinney.id.au> Edward Montague writes: > I'd like to eventually have 3D graphics within an application For that requirement, your application will need to make use of a library for presenting and interacting with 3D objects. To my knowledge there is no such thing in the standard library, so you'll need to bring in a third-party distribution for that. > constructed through a GUI Designer ; preferably with quality > approaching or exceeding that of MayaVI2. I haven't used Blender for a separate Python program, but I know that people like its Python API . -- \ ?Most people are other people. Their thoughts are someone | `\ else?s opinions, their lives a mimicry, their passions a | _o__) quotation.? ?Oscar Wilde, _De Profundis_, 1897 | Ben Finney From tjreedy at udel.edu Mon Jun 26 02:03:21 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 26 Jun 2017 02:03:21 -0400 Subject: comments and the continuation prompt In-Reply-To: <85k23zs9t1.fsf@benfinney.id.au> References: <59506b95$0$1611$c3e8da3$5496439d@news.astraweb.com> <85k23zs9t1.fsf@benfinney.id.au> Message-ID: On 6/25/2017 11:32 PM, Ben Finney wrote: > Steve D'Aprano writes: > >> On Mon, 26 Jun 2017 08:44 am, Stefan Ram wrote: >> >>> According to The Python Language Reference Release 3.6.0, 2.1.3 >>> Comments, ?A comment signifies the end of the logical line unless >>> the implicit line joining rules are invoked.?. >>> >>> So, why do I get a continuation prompt when I enter a comment? In IDLE, you don't. >>> # >>> #sjflksj >>> Maybe this was once true for the interactive interpreter and changed. Or maybe this is buglet in IDLE in terms of imitating the console interpreter. I have no idea. >> Why not? As far as the interactive interpreter is concerned, you >> haven't yet entered a statement. > > And yet, according to the Language Reference, the logical line has ended > and another begun. > > So I think the question is worth exploring: Why does the interactive > prompt imply the logical line is continuing, when the Language Reference > would say otherwise? > > Maybe the answer is ?the continuation prompt does not prompt for the > continuation of a logical line, but the continuation of else>?. > > What exactly goes in the ?? placeholder; that is, > exactly what should the user understand by that transition from one > prompt to a different one? Look into the behavior of compile('code', 'fake', 'single'). -- Terry Jan Reedy From dhariyalbhaskar at gmail.com Mon Jun 26 03:19:16 2017 From: dhariyalbhaskar at gmail.com (Bhaskar Dhariyal) Date: Mon, 26 Jun 2017 00:19:16 -0700 (PDT) Subject: Unable to apply stop words in Pandas dataframe Message-ID: <099fb884-fdbe-471d-92ed-e799e9271d97@googlegroups.com> Hi everyone! I have a dataset which I want to make model trainable. I ahve been trying to do some thing for past 2-3 days. Actually I wanted to clean 'desc' and 'keywords' column from the dataset. I am using NLTK to vectorize, than remove stopwords & alpha numeric values and do stemming. Moreover desc column also contain NULL values. But the solution I reached was on 2D list, to which I was not able to convert single pandas series, so that I could change change it word vector. Please tell me hoe to do this using Pandas, I searched lot in stackoverflow also but no results. Link to dataset: https://drive.google.com/open?id=0B1D4AyluMGU0MGVkbWdvQkM3aUk Thanks in advance Bhaskar From ikorot01 at gmail.com Mon Jun 26 07:11:13 2017 From: ikorot01 at gmail.com (Igor Korot) Date: Mon, 26 Jun 2017 07:11:13 -0400 Subject: GUI Designer[s] In-Reply-To: <85fuens75t.fsf@benfinney.id.au> References: <85fuens75t.fsf@benfinney.id.au> Message-ID: Hi, On Mon, Jun 26, 2017 at 12:29 AM, Ben Finney wrote: > Edward Montague writes: > >> I'd like to eventually have 3D graphics within an application > > For that requirement, your application will need to make use of a > library for presenting and interacting with 3D objects. > > To my knowledge there is no such thing in the standard library, so > you'll need to bring in a third-party distribution for that. > >> constructed through a GUI Designer ; preferably with quality >> approaching or exceeding that of MayaVI2. > > I haven't used Blender for a separate > Python program, but I know that people like its Python API > . Take a look at OpenGL and its wx implementation. Thank you. > > -- > \ ?Most people are other people. Their thoughts are someone | > `\ else?s opinions, their lives a mimicry, their passions a | > _o__) quotation.? ?Oscar Wilde, _De Profundis_, 1897 | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list From steve+python at pearwood.info Mon Jun 26 08:53:06 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 26 Jun 2017 22:53:06 +1000 Subject: comments and the continuation prompt References: <59506b95$0$1611$c3e8da3$5496439d@news.astraweb.com> <85k23zs9t1.fsf@benfinney.id.au> Message-ID: <595103b5$0$1622$c3e8da3$5496439d@news.astraweb.com> On Mon, 26 Jun 2017 01:32 pm, Ben Finney wrote: > Steve D'Aprano writes: > >> On Mon, 26 Jun 2017 08:44 am, Stefan Ram wrote: >> >> > According to The Python Language Reference Release 3.6.0, 2.1.3 >> > Comments, ?A comment signifies the end of the logical line unless >> > the implicit line joining rules are invoked.?. >> > >> > So, why do I get a continuation prompt when I enter a comment? >> >> Why not? As far as the interactive interpreter is concerned, you >> haven't yet entered a statement. > > And yet, according to the Language Reference, the logical line has ended > and another begun. Correct. That doesn't contradict anything I said. > So I think the question is worth exploring: Why does the interactive > prompt imply the logical line is continuing, when the Language Reference > would say otherwise? What leads you to believe that the prompt implies the logical line is continuing? That's not implied by the prompt. I gave examples where the secondary prompt is displayed that have nothing to do with logical lines: py> def func(): ... x = 1 ... y = 2 ... That's not one logical line, and the presence of the secondary prompt ps2 does not imply that it is. The same applies to class, try, for, while, with, triple-quoted strings, and possibly others. So why are we (by which I mean *not me*) thinking that the ps2 secondary prompt implies a single logical line? All logical line continuations are displayed with the secondary prompt. This does not imply that every time the secondary prompt is displayed, the line is being continued. Analogy: all people called "Ben" are human beings, but not all human beings are called "Ben". There is no puzzle here. The secondary prompt displayed after a comment is a quirk of the CPython REPL, it isn't part of the language and it doesn't need to be fixed because it isn't broken. - IDLE doesn't do this. - IPython doesn't do this: In [1]: #comment In [2]: - Jython 2.5 doesn't do this: >>> # comment >>> - Neither does bpython. - I no longer have IronPython installed on any of my systems, so I can't check that, or PyPy. - The emulated REPL provided by the `code` module doesn't do it either: py> import code py> code.interact() Python 3.5.2 (default, Oct 12 2016, 10:47:40) [GCC 4.1.2 20080704 (Red Hat 4.1.2-55)] on linux Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) py> # comment py> It's a quirk of the CPython default REPL, of no consequence and with no implications, and it especially has nothing to do with line continuations. > Maybe the answer is ?the continuation prompt does not prompt for the > continuation of a logical line, Sometimes it does. > but the continuation of ?. > > What exactly goes in the ?? placeholder; that is, > exactly what should the user understand by that transition from one > prompt to a different one? A nested block; a triple-quoted string; a line continuation; a statement after a comment; whatever reasonable thing a REPL chooses. The language does not, so far as I am aware, mandate when the ps1 and ps2 prompts are used, and there is no basis for drawing any conclusion about the Python language from the choices made by any specific REPL -- not even the default CPython REPL. https://docs.python.org/3/library/sys.html#sys.ps1 -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Mon Jun 26 08:58:54 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 26 Jun 2017 22:58:54 +1000 Subject: comments and the continuation prompt In-Reply-To: <595103b5$0$1622$c3e8da3$5496439d@news.astraweb.com> References: <59506b95$0$1611$c3e8da3$5496439d@news.astraweb.com> <85k23zs9t1.fsf@benfinney.id.au> <595103b5$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Jun 26, 2017 at 10:53 PM, Steve D'Aprano wrote: > - Jython 2.5 doesn't do this: > >>>> # comment >>>> > > - Neither does bpython. > > - I no longer have IronPython installed on any of my systems, so I can't check > that, or PyPy. PyPy doesn't, fwiw. ChrisA From as at sci.fi Mon Jun 26 09:07:10 2017 From: as at sci.fi (Anssi Saari) Date: Mon, 26 Jun 2017 16:07:10 +0300 Subject: comp.lang.python killfile rule References: Message-ID: John Black writes: > All, in case this is useful to anyone, this rule that tells my newsreader > which posts to kill really cleans up the group. I get by just with a very old rule that lowers the score of articles where the subject is in all caps. Those articles end up in the bottom of the article list with the other crap and are easily ignored. From guillaume.paulet at giome.fr Mon Jun 26 11:04:47 2017 From: guillaume.paulet at giome.fr (guillaume.paulet at giome.fr) Date: Mon, 26 Jun 2017 17:04:47 +0200 Subject: =?UTF-8?Q?Release_of_Scalpl_=28v0=2E2=2E5=29_=E2=9C=A8?= =?UTF-8?Q?=F0=9F=8D=B0=E2=9C=A8_-_a_lightweight_wrapper_for_your_nested_d?= =?UTF-8?Q?ictionaries?= Message-ID: <8ef111de35d34f86b75070fd4892f300@giome.fr> Hi everyone ! I released a new version (0.2.5) of **Scalpl** which is available on PyPI :) https://github.com/ducdetronquito/scalpl You can install it via pip: pip3 install scalpl Scalpl is a lightweight wrapper that helps you to operate on nested dictionaries through the built-in dict API, by using dot-separated string keys. You might find it useful when working with document-oriented database queries, REST APIs, configuration files, etc... It's not a drop-in replacement for your dictionaries, just syntactic sugar to avoid this['annoying']['kind']['of']['things'] and prefer['a.different.approach']. The benefits of Scalpl are the following: * Faster than addict or Box. * Allows you to use the entire dict API 'with.this.kind.of.keys'. * Almost no instantiation/conversion cost, it's just a wrapper. This new release (0.2.5) is just a small API improvement. In the previous version of Scalpl, if you wanted to iterate a list of dictionaries and and operate on it, you would have done the following: ``` data = { 'pokemons': [ { 'name': 'Bulbasaur', 'type': ['Grass', 'Poison'], 'category': 'Seed', 'ability': 'Overgrow' }, { 'name': 'Charmander', 'type': 'Fire', 'category': 'Lizard', 'ability': 'Blaze', }, { 'name': 'Squirtle', 'type': 'Water', 'category': 'Tiny Turtle', 'ability': 'Torrent', } ], 'trainers': [ { 'name': 'Ash', 'hometown': 'Pallet Town' } ] } proxy = Cut(data) pokemons = proxy['pokemons'] for pokemon in Cut.all(pokemons): pokemon.setdefault('moves.Scratch', {'power': 40}) ``` Now, the API allows you to provied composite key directly to the Cut.all method: ``` for pokemon in proxy.all('pokemons'): pokemon.setdefault('moves.Scratch', {'power': 40}) ``` Do not hesitate to give me feedbacks on the module itself, it is one of my first public project ! Have a great afternoon :) From bxstover at yahoo.co.uk Mon Jun 26 11:16:29 2017 From: bxstover at yahoo.co.uk (Ben S.) Date: Mon, 26 Jun 2017 08:16:29 -0700 (PDT) Subject: Syntax error for simple script Message-ID: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> Sorry for this newbie question: I installed Python v3.6.1 on win 7. Afterwards I tried to execute the following simple python script from webpage http://www.pythonforbeginners.com/code-s...me-script/: Python Code: from datetime import datetime now = datetime.now() mm = str(now.month) dd = str(now.day) yyyy = str(now.year) hour = str(now.hour) mi = str(now.minute) ss = str(now.second) print mm + "/" + dd + "/" + yyyy + " " + hour + ":" + mi + ":" + ss When executing D:\tools\python\python.exe dateParser.py it yields a syntax error: File "dateParser.py", line 17 print mm + "/" + dd + "/" + yyyy + " " + hour + ":" + mi + ":" + ss ^ SyntaxError: Missing parentheses in call to 'print' Whats wrong? Thank you Peter From steve+python at pearwood.info Mon Jun 26 11:42:22 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 27 Jun 2017 01:42:22 +1000 Subject: Syntax error for simple script References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> Message-ID: <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> On Tue, 27 Jun 2017 01:16 am, Ben S. wrote: > print mm + "/" + dd + "/" + yyyy + " " + hour + ":" + mi + ":" + ss > ^ > SyntaxError: Missing parentheses in call to 'print' > > Whats wrong? Did you read the error message? Missing parentheses in call to 'print' Is the message not clear enough? The call to print requires parentheses (round brackets), like all other functions. print(mm + "/" + dd + "/" + yyyy + " " + hour + ":" + mi + ":" + ss) If the error message isn't clear enough, what can we do to make it more clear? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rurpy at yahoo.com Mon Jun 26 12:41:31 2017 From: rurpy at yahoo.com (rurpy at yahoo.com) Date: Mon, 26 Jun 2017 09:41:31 -0700 (PDT) Subject: Syntax error for simple script In-Reply-To: <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> On 06/26/2017 09:42 AM, Steve D'Aprano wrote: > On Tue, 27 Jun 2017 01:16 am, Ben S. wrote: > >> print mm + "/" + dd + "/" + yyyy + " " + hour + ":" + mi + ":" + ss >> ^ >> SyntaxError: Missing parentheses in call to 'print' >> >> Whats wrong? > > Did you read the error message? > > Missing parentheses in call to 'print' > > Is the message not clear enough? The call to print requires parentheses (round > brackets), like all other functions. > > print(mm + "/" + dd + "/" + yyyy + " " + hour + ":" + mi + ":" + ss) > > > If the error message isn't clear enough, what can we do to make it more clear? How about: In Python 2, 'print' was a statement and did not require parenthesis around its argument. In Python 3 'print' has been changed to a function and now, like all functions, requires parenthesis around its arguments: python 2: print arg1, arg2,... python 3: print (arg1, arg2,...) Did you seriously not understand how someone, reading a tutorial or book written for Python 2, might be confused when a specific simple syntax he/she was explicitly told works, doesn't work? Not everyone new to and starting to explore Python appreciates (?!) the major compatibility breaks that occurred between Py2 and Py3. One would expect that in such a situation, the new user would certainly question his/her code rather than something changed in Python. Or better than changing the message, how about leaving it alone and simply responding helpfully rather than snappily and condescending to inevitable confusion that surely must have been anticipated when the Python developers chose to make this change. It's no wonder that some people (often women) who don't like contention, find this list rather hostile. From none at invalid.com Mon Jun 26 13:00:57 2017 From: none at invalid.com (mm0fmf) Date: Mon, 26 Jun 2017 18:00:57 +0100 Subject: Syntax error for simple script In-Reply-To: <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: On 26/06/2017 17:41, rurpy at yahoo.com wrote: > On 06/26/2017 09:42 AM, Steve D'Aprano wrote: >> On Tue, 27 Jun 2017 01:16 am, Ben S. wrote: >> >>> print mm + "/" + dd + "/" + yyyy + " " + hour + ":" + mi + ":" + ss >>> ^ >>> SyntaxError: Missing parentheses in call to 'print' >>> >>> Whats wrong? >> >> Did you read the error message? >> >> Missing parentheses in call to 'print' >> >> Is the message not clear enough? The call to print requires parentheses (round >> brackets), like all other functions. >> >> print(mm + "/" + dd + "/" + yyyy + " " + hour + ":" + mi + ":" + ss) >> >> >> If the error message isn't clear enough, what can we do to make it more clear? > > How about: > > In Python 2, 'print' was a statement and did not require > parenthesis around its argument. In Python 3 'print' has > been changed to a function and now, like all functions, > requires parenthesis around its arguments: > python 2: print arg1, arg2,... > python 3: print (arg1, arg2,...) > > Did you seriously not understand how someone, reading a tutorial or > book written for Python 2, might be confused when a specific simple > syntax he/she was explicitly told works, doesn't work? Not everyone > new to and starting to explore Python appreciates (?!) the major > compatibility breaks that occurred between Py2 and Py3. One would > expect that in such a situation, the new user would certainly question > his/her code rather than something changed in Python. > > Or better than changing the message, how about leaving it alone and > simply responding helpfully rather than snappily and condescending to > inevitable confusion that surely must have been anticipated when the > Python developers chose to make this change. > > It's no wonder that some people (often women) who don't like contention, > find this list rather hostile. > Sorry to victim blame but why can't people copy the error message into a search engine, such as Google, and see what the problem is themselves? The first page of results gives plenty of detail of the differences between Python 2 & 3 and this print issue with lots of explanations about print vs. print(). I'd only start asking questions on a newsgroup if I couldn't find explanations on the web or having read the explanations they were all telling me different things. YMMV From skip.montanaro at gmail.com Mon Jun 26 13:15:59 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 26 Jun 2017 12:15:59 -0500 Subject: Syntax error for simple script In-Reply-To: <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: On Mon, Jun 26, 2017 at 11:41 AM, Rurpy via Python-list wrote: > In Python 2, 'print' was a statement and did not require > parenthesis around its argument. In Python 3 'print' has > been changed to a function I'm not sure that at this point Python 3's error messages should be polluted with explanations for the difference between 2.x and 3.x. Skip From bill at baddogconsulting.com Mon Jun 26 13:23:45 2017 From: bill at baddogconsulting.com (Bill Deegan) Date: Mon, 26 Jun 2017 10:23:45 -0700 Subject: Question about propagating universal_newlines through subprocess.Popen to io.TextIOWrapper Message-ID: Greetings, I was surprised to see that if I set encoding in my call to subprocess.Popen() as follows: p = Popen(cmd, stdin=stdin, stdout=subprocess.PIPE, stderr=stderr_value, env=os.environ, universal_newlines=False, #universal_newlines, encoding='utf-8') That universal_newlines value is discarded due to: text_mode = encoding or errors or universal_newlines ... if text_mode: self.stdout = io.TextIOWrapper(self.stdout, encoding=encoding, errors=errors) There doesn't seem to be a way to set encoding without forcing univeral_newlines. This seems like a bug? -Bill From rosuav at gmail.com Mon Jun 26 13:30:46 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Jun 2017 03:30:46 +1000 Subject: Syntax error for simple script In-Reply-To: <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: On Tue, Jun 27, 2017 at 2:41 AM, Rurpy via Python-list wrote: > How about: > > In Python 2, 'print' was a statement and did not require > parenthesis around its argument. In Python 3 'print' has > been changed to a function and now, like all functions, > requires parenthesis around its arguments: > python 2: print arg1, arg2,... > python 3: print (arg1, arg2,...) Absolutely, if we consider that Python 2 is the official standard and Python 3 is a buggy interloper. But if Python 3 is the standard and Python 2 is the legacy version, then the existing message is far more appropriate, without being unhelpful. ChrisA From rosuav at gmail.com Mon Jun 26 13:31:57 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Jun 2017 03:31:57 +1000 Subject: Syntax error for simple script In-Reply-To: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> Message-ID: On Tue, Jun 27, 2017 at 1:16 AM, Ben S. via Python-list wrote: > I installed Python v3.6.1 on win 7. Afterwards I tried to execute the following simple python script from webpage http://www.pythonforbeginners.com/code-s...me-script/: Can you provide the complete link please? Something seems to have ellipsized it. Whatever the page is, it needs to be updated to be Python 3 compatible. ChrisA From jblack at nopam.com Mon Jun 26 14:32:43 2017 From: jblack at nopam.com (John Black) Date: Mon, 26 Jun 2017 13:32:43 -0500 Subject: comp.lang.python killfile rule References: Message-ID: In article , miragewebstudio12 at gmail.com says... > Just felt like posting, wouldn't it be pythonic if it was > if word in [list]: > ignore > > Save time and easily maintainable Yes for readers that can interpret Python. Mine doesn't. This is the rule per the syntax of my reader but as you say that is easily translated to any other format. John Black From best_lay at yahoo.com Mon Jun 26 15:06:26 2017 From: best_lay at yahoo.com (Wildman) Date: Mon, 26 Jun 2017 14:06:26 -0500 Subject: Syntax error for simple script References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> Message-ID: On Tue, 27 Jun 2017 03:31:57 +1000, Chris Angelico wrote: > On Tue, Jun 27, 2017 at 1:16 AM, Ben S. via Python-list > wrote: >> I installed Python v3.6.1 on win 7. Afterwards I tried to execute the following simple python script from webpage http://www.pythonforbeginners.com/code-s...me-script/: > > Can you provide the complete link please? Something seems to have > ellipsized it. Whatever the page is, it needs to be updated to be > Python 3 compatible. > > ChrisA Here it is... http://www.pythonforbeginners.com/code-snippets-source-code/date-and-time-script/ -- GNU/Linux user #557453 The cow died so I don't need your bull! From rosuav at gmail.com Mon Jun 26 15:22:48 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Jun 2017 05:22:48 +1000 Subject: Syntax error for simple script In-Reply-To: References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> Message-ID: On Tue, Jun 27, 2017 at 5:06 AM, Wildman via Python-list wrote: > Here it is... > > http://www.pythonforbeginners.com/code-snippets-source-code/date-and-time-script/ Since it's not world-editable, I've posted a comment. But nobody ever reads those :| I strongly recommend learning Python from a course/book/website that is Python 3 compatible. ChrisA From torriem at gmail.com Mon Jun 26 15:28:20 2017 From: torriem at gmail.com (Michael Torrie) Date: Mon, 26 Jun 2017 13:28:20 -0600 Subject: GUI Designer[s] In-Reply-To: References: <85fuens75t.fsf@benfinney.id.au> Message-ID: On 06/26/2017 05:11 AM, Igor Korot wrote: > Take a look at OpenGL and its wx implementation. Or PyQt5 with its integrated OpenGL support. Alternatively, there's QtQuick and its own OpenGL scene management system. From eryksun at gmail.com Mon Jun 26 15:44:21 2017 From: eryksun at gmail.com (eryk sun) Date: Mon, 26 Jun 2017 19:44:21 +0000 Subject: Question about propagating universal_newlines through subprocess.Popen to io.TextIOWrapper In-Reply-To: References: Message-ID: On Mon, Jun 26, 2017 at 5:23 PM, Bill Deegan wrote: > > That universal_newlines value is discarded due to: > > text_mode = encoding or errors or universal_newlines > > ... > > if text_mode: > self.stdout = io.TextIOWrapper(self.stdout, > encoding=encoding, errors=errors) > > There doesn't seem to be a way to set encoding without forcing > univeral_newlines. > > This seems like a bug? The behavior is documented: If encoding or errors are specified, or universal_newlines is true, the file objects stdin, stdout and stderr will be opened in text mode using the encoding and errors specified in the call or the defaults for io.TextIOWrapper. For stdin, line ending characters '\n' in the input will be converted to the default line separator os.linesep. For stdout and stderr, all line endings in the output will be converted to '\n'. For more information see the documentation of the io.TextIOWrapper class when the newline argument to its constructor is None. Prior to 3.6, the way to get text streams was to enable universal_newlines. Maybe for 3.7 the default can change to None, with the addition of the following code: if universal_newlines is None or universal_newlines: newline = None else: newline = '' if text_mode: self.stdin = io.TextIOWrapper(self.stdin, write_through=True, line_buffering=(bufsize == 1), encoding=encoding, errors=errors, newline=newline) From phnetynka at gmail.com Mon Jun 26 16:18:15 2017 From: phnetynka at gmail.com (phnetynka at gmail.com) Date: Mon, 26 Jun 2017 13:18:15 -0700 (PDT) Subject: Deadline Extension - CfP: 14th International Conference on Managed Languages & Runtimes Message-ID: <64226516-5703-4958-99e1-4bc59c86685d@googlegroups.com> CALL FOR PAPERS - ManLang '17 14th International Conference on Managed Languages & Runtimes (ManLang, formerly PPPJ) September 25-29, 2017, Prague, Czech Republic http://d3s.mff.cuni.cz/conferences/manlang17/ ********************************************************** SUBMISSION DEADLINE EXTENDED TO JULY 3, 2017 ********************************************************** Sponsored by Charles University and Oracle Labs In-cooperation with ACM SIGAPP and SIGPLAN ManLang is a premier forum for presenting and discussing innovations and breakthroughs in the area of programming languages and runtime systems, which form the basis of many modern computing systems, from small scale (embedded and real-time systems) to large-scale (cloud-computing and big-data platforms). ------------------------------------------------------------------------------- TOPICS - Virtual machines * Managed runtime systems (JVM, Dalvik VM and Android Runtime (ART), LLVM, .NET CLR, RPython, etc.) * VM design and optimization * VMs for mobile and embedded devices * VMs for real-time applications * Isolation and resource control * Memory management - Languages * Managed languages (Java, Scala, JavaScript, Python, Ruby, C#, F#, Clojure, Groovy, Kotlin, R, Smalltalk, Racket, etc.) * Domain-specific languages * Language design and calculi * Compilers * Language interoperability * Parallelism and concurrency * Modular and aspect-oriented programming * Model-driven development * Frameworks and applications * Teaching - Techniques and tools * Static and dynamic program analysis * Real-time systems * Embedded systems * Testing * Verification * Monitoring and debugging * Security and information flow * Workload characterization and performance evaluation Do not hesitate to contact the Program Chair (Shigeru Chiba ) to clarify if a particular topic falls within the scope of ManLang '17. ------------------------------------------------------------------------------- IMPORTANT DATES Abstract submission (not required but recommended): Monday June 26, 2017 (AoE) Paper submission (extended): Monday July 3, 2017 (AoE) Paper acceptance notification: Monday August 7, 2017 Poster submission: Wednesday August 9, 2017 Poster acceptance notification: Wednesday August 16, 2017 Camera ready version due: Monday August 21, 2017 ------------------------------------------------------------------------------- SUBMISSIONS ManLang '17 accepts four types of submissions: - Regular research paper: up to 12 pages - Work-in-progress paper: up to 6 pages - Industry and tool paper: up to 6 pages - Poster (standalone or accompanying paper submission) The conference proceedings will be published as part of the ACM International Conference Proceedings Series and will be disseminated through the ACM Digital Library. Research papers will be judged on their relevance, novelty, technical rigor, and contribution to the state-of-the-art. For work-in-progress research papers, more emphasis will be placed on novelty and the potential of the new idea than on technical rigor and experimental results. Industry and tool papers will be judged on their relevance, usefulness, and results. Suitability for demonstration and availability will also be considered for tool papers. Posters can accompany any submission, in particular to provide additional demonstration and discussion opportunities. Criteria for standalone posters will be similar to criteria for short papers. ------------------------------------------------------------------------------- PROGRAM CHAIR * Shigeru Chiba, University of Tokyo, Japan GENERAL CHAIR * Petr Tuma, Charles University, Czech Republic PROGRAM COMMITTEE * Walter Binder, University of Lugano (USI), Switzerland * Christoph Bockisch, University Marburg, Germany * Elisa Gonzalez Boix, Vrije Universiteit Brussel, Belgium * Carl Friedrich Bolz, PyPy * Walter Cazzola, Universit? degli Studi di Milano, Italy * Yvonne Coady, University of Victoria, Canada * St?phane Ducasse, Inria Lille Nord Europe, France * G?rel Hedin, Lunds University, Sweden * Robert Hirschfeld, HPI, Germany * Tony Hosking, Purdue University, USA * Doug Lea, State University of New York at Oswego, USA * J. Eliot Moss, University of Massachusetts Amherst, USA * Hanspeter M?ssenb?ck, Johannes Kepler University Linz, Austria * Tiark Rompf, Purdue University, USA * Koichi Sasada, Cookpad Inc., Japan * Martin Schoeberl, Technical University of Denmark * Jeremy Singer, University of Glasgow, Scotland * Vincent St-Amour, Northwestern University, USA * Laurence Tratt, King's College London, UK * Jan Vitek, Northeastern University, USA * Christian Wimmer, Oracle Labs, USA * Jianjun Zhao, Kyushu University, Japan From bill at baddogconsulting.com Mon Jun 26 16:59:25 2017 From: bill at baddogconsulting.com (Bill Deegan) Date: Mon, 26 Jun 2017 13:59:25 -0700 Subject: Question about propagating universal_newlines through subprocess.Popen to io.TextIOWrapper In-Reply-To: References: Message-ID: On Mon, Jun 26, 2017 at 12:44 PM, eryk sun wrote: > On Mon, Jun 26, 2017 at 5:23 PM, Bill Deegan > wrote: > > > > That universal_newlines value is discarded due to: > > > > text_mode = encoding or errors or universal_newlines > > > > ... > > > > if text_mode: > > self.stdout = io.TextIOWrapper(self.stdout, > > encoding=encoding, errors=errors) > > > > There doesn't seem to be a way to set encoding without forcing > > univeral_newlines. > > > > This seems like a bug? > > The behavior is documented: > > If encoding or errors are specified, or universal_newlines is true, > the file objects stdin, stdout and stderr will be opened in text > mode using the encoding and errors specified in the call or the > defaults for io.TextIOWrapper. > > For stdin, line ending characters '\n' in the input will be > converted to the default line separator os.linesep. For stdout and > stderr, all line endings in the output will be converted to '\n'. > For more information see the documentation of the io.TextIOWrapper > class when the newline argument to its constructor is None. > > Prior to 3.6, the way to get text streams was to enable > universal_newlines. Maybe for 3.7 the default can change to None, with > the addition of the following code: > > if universal_newlines is None or universal_newlines: > newline = None > else: > newline = '' > > if text_mode: > self.stdin = io.TextIOWrapper(self.stdin, write_through=True, > line_buffering=(bufsize == 1), > encoding=encoding, errors=errors, newline=newline) > Ideally (for my use case) it would be something which propagated universal_newlines to io.TextIOWrapper().. rather than discards it. In my case I want the stdout to be encoded utf-8, but I do not want \r's changed to \n's as my test system is capturing the output of a progress indicator which uses \r to return to beginning of line and overwrite the previous output. -Bill From eryksun at gmail.com Mon Jun 26 17:22:53 2017 From: eryksun at gmail.com (eryk sun) Date: Mon, 26 Jun 2017 21:22:53 +0000 Subject: Question about propagating universal_newlines through subprocess.Popen to io.TextIOWrapper In-Reply-To: References: Message-ID: On Mon, Jun 26, 2017 at 8:59 PM, Bill Deegan wrote: > > Ideally (for my use case) it would be something which propagated > universal_newlines to io.TextIOWrapper().. rather than discards it. > In my case I want the stdout to be encoded utf-8, but I do not want \r's > changed to \n's as my test system is capturing the output of a progress > indicator which uses \r to return to beginning of line and overwrite the > previous output. An enhancement issue can be created for 3.7, but this is new behavior that certainly won't be backported to 3.6. For now you'll have to do it yourself. Leave out the encoding argument, and manually wrap p.stdout with an io.TextIOWrapper that sets the newline argument to an empty string. From bill at baddogconsulting.com Mon Jun 26 17:57:26 2017 From: bill at baddogconsulting.com (Bill Deegan) Date: Mon, 26 Jun 2017 14:57:26 -0700 Subject: Question about propagating universal_newlines through subprocess.Popen to io.TextIOWrapper In-Reply-To: References: Message-ID: On Mon, Jun 26, 2017 at 2:22 PM, eryk sun wrote: > On Mon, Jun 26, 2017 at 8:59 PM, Bill Deegan > wrote: > > > > Ideally (for my use case) it would be something which propagated > > universal_newlines to io.TextIOWrapper().. rather than discards it. > > In my case I want the stdout to be encoded utf-8, but I do not want \r's > > changed to \n's as my test system is capturing the output of a progress > > indicator which uses \r to return to beginning of line and overwrite the > > previous output. > > An enhancement issue can be created for 3.7, but this is new behavior > that certainly won't be backported to 3.6. For now you'll have to do > it yourself. Leave out the encoding argument, and manually wrap > p.stdout with an io.TextIOWrapper that sets the newline argument to an > empty string. > O.k. I'll file an enhancement. Seems like propagating the values would be better functionality. Thanks! -Bill From python at lucidity.plus.com Mon Jun 26 18:34:06 2017 From: python at lucidity.plus.com (Erik) Date: Mon, 26 Jun 2017 23:34:06 +0100 Subject: Syntax error for simple script In-Reply-To: References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: On 26/06/17 18:30, Chris Angelico wrote: > On Tue, Jun 27, 2017 at 2:41 AM, Rurpy via Python-list > wrote: >> How about: >> >> In Python 2, 'print' was a statement and did not require >> parenthesis around its argument. In Python 3 'print' has >> been changed to a function and now, like all functions, >> requires parenthesis around its arguments: >> python 2: print arg1, arg2,... >> python 3: print (arg1, arg2,...) > > Absolutely, if we consider that Python 2 is the official standard and > Python 3 is a buggy interloper. But if Python 3 is the standard and > Python 2 is the legacy version, then the existing message is far more > appropriate, without being unhelpful. To be fair, this already seems to be a special case: ----------------------------------------------------------- $ python3 Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> def foo(s): return s ... >>> type(foo) >>> type(enumerate) >>> type(len) >>> type(print) >>> foo "bar" File "", line 1 foo "bar" ^ SyntaxError: invalid syntax >>> enumerate "bar" File "", line 1 enumerate "bar" ^ SyntaxError: invalid syntax >>> len "bar" File "", line 1 len "bar" ^ SyntaxError: invalid syntax >>> print "bar" File "", line 1 print "bar" ^ SyntaxError: Missing parentheses in call to 'print' >>> p = print >>> p "foo" File "", line 1 p "foo" ^ SyntaxError: invalid syntax >>> ----------------------------------------------------------- If that's the case, then why make it so terse? It's presumably there as a special case specifically to tell/remind people that print became a function in Py3. I think the suggestion above is a bit wordy, but even if it was something like "Missing parentheses in call to the 'print' function" or even "Missing parentheses in call to 'print'. Using 'print' as a statement is obsolete syntax in Python 3." ... or whatever is considered more descriptive. Either that or just make it "SyntaxError: invalid syntax" like all the others. It seems to fall somewhere between the two at the moment - it's trying to be more helpful but just falls short for a beginner (if one already knows the difference and has just forgotten to call it correctly, it's a nice *reminder*, so I'd actually prefer it not to just become the basic "invalid syntax" message). E. From python at lucidity.plus.com Mon Jun 26 19:39:09 2017 From: python at lucidity.plus.com (Erik) Date: Tue, 27 Jun 2017 00:39:09 +0100 Subject: Syntax error for simple script In-Reply-To: References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: <52e9646f-9ffc-6cfb-e746-9c98e9174430@lucidity.plus.com> On 27/06/17 00:21, Stefan Ram wrote: > Erik writes: Using 'print' as a >> statement is obsolete syntax in Python 3. > > ?print? never was a statement. > ?print? was a keyword (2.7.6 2.3.1). *sigh* Whatever. You have completely ignored my point (that perhaps the message could be a little more descriptive, as it's a special case anyway) and instead are trying to prove me wrong on some technicality because I used one word incorrectly. Fine, I used the wrong term. How does that help this conversation about the usefulness of the error message? OK, so "Using 'print' as a keyword is obsolete syntax" then? I even said I didn't care what the wording was, just that it could be a tad more descriptive. E. From miki.tebeka at gmail.com Tue Jun 27 00:21:24 2017 From: miki.tebeka at gmail.com (Miki Tebeka) Date: Mon, 26 Jun 2017 21:21:24 -0700 (PDT) Subject: Unable to apply stop words in Pandas dataframe In-Reply-To: <099fb884-fdbe-471d-92ed-e799e9271d97@googlegroups.com> References: <099fb884-fdbe-471d-92ed-e799e9271d97@googlegroups.com> Message-ID: Can you show us some of the code you tried? On Monday, June 26, 2017 at 10:19:46 AM UTC+3, Bhaskar Dhariyal wrote: > Hi everyone! > > I have a dataset which I want to make model trainable. I ahve been trying to do some thing for past 2-3 days. > > Actually I wanted to clean 'desc' and 'keywords' column from the dataset. I am using NLTK to vectorize, than remove stopwords & alpha numeric values and do stemming. Moreover desc column also contain NULL values. But the solution I reached was on 2D list, to which I was not able to convert single pandas series, so that I could change change it word vector. > > Please tell me hoe to do this using Pandas, I searched lot in stackoverflow also but no results. > > Link to dataset: https://drive.google.com/open?id=0B1D4AyluMGU0MGVkbWdvQkM3aUk > > Thanks in advance > > Bhaskar From rustompmody at gmail.com Tue Jun 27 00:23:16 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 26 Jun 2017 21:23:16 -0700 (PDT) Subject: Syntax error for simple script In-Reply-To: <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: <1eeda44b-525d-40e5-949b-46978003e68d@googlegroups.com> On Monday, June 26, 2017 at 10:11:46 PM UTC+5:30, rurpy wrote: > How about: ?suggested error message? > Or better than changing the message, how about leaving it alone and > simply responding helpfully? Since everyone seems to only have read the first suggestion from rurpy, let me ask, paraphrasing Steven's > If the error message isn't clear enough, what can we do to make it more clear? How better could rurpy have written "Or better" so that others could see the preferred ordering of the two suggestions? From steve+python at pearwood.info Tue Jun 27 01:37:27 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 27 Jun 2017 15:37:27 +1000 Subject: Syntax error for simple script References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: <5951ef19$0$1595$c3e8da3$5496439d@news.astraweb.com> On Tue, 27 Jun 2017 09:21 am, Stefan Ram wrote: > Erik writes: Using 'print' as a >>statement is obsolete syntax in Python 3. > > ?print? never was a statement. Of course it could be a statement, just like "y = x + 1" is a statement. > ?print? was a keyword (2.7.6 2.3.1). It can be both, depending on context. > What /was/ a statement in 2.7.6 was (2.7.6 6.6) > > "print" ( [ expression ("," expression )* ["," ]]| > ">>" expression [( "," expression )+ [ "," ]]) Which includes the case that print is called with no arguments: print which is a statement. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bxstover at yahoo.co.uk Tue Jun 27 01:41:11 2017 From: bxstover at yahoo.co.uk (Ben S.) Date: Mon, 26 Jun 2017 22:41:11 -0700 (PDT) Subject: "Python launcher" required to run *.py scripts on Windows? Message-ID: As I observed v3.6.1 installs (on Windows 7) in addition to the core python engine a second program "Python Launcher". As far as I read this component seems to be not necessary since it only aims to facilitate the handling with *.py scripts on Windows. When I always call Python script from CommandPrompt like D:\tools\Python\python.exe mypythonscript.py then this Launcher should not be necessary. Am I right? Since I prefer to have as less prgrams installed as possible I consider to uninstall the Python Launcher. Is this possible without causing trouble to Python executions? Ben From tjreedy at udel.edu Tue Jun 27 03:14:22 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 27 Jun 2017 03:14:22 -0400 Subject: "Python launcher" required to run *.py scripts on Windows? In-Reply-To: References: Message-ID: On 6/27/2017 1:41 AM, Ben S. via Python-list wrote: > As I observed v3.6.1 installs (on Windows 7) in addition to the core python engine a second program "Python Launcher". > > As far as I read this component seems to be not necessary since it only aims to facilitate the handling with *.py scripts on Windows. > > When I always call Python script from CommandPrompt like > > D:\tools\Python\python.exe mypythonscript.py > > then this Launcher should not be necessary. > > Am I right? As long as you have only 1 version of python installed, then there is no ambiguity to 'python', and you do not really need it. But it is nice to be able to type 'py' regardless of the current working directory. If you have more than 1 python version installed, it is very handy. > Since I prefer to have as less prgrams installed as possible I consider to uninstall the Python Launcher. > > Is this possible without causing trouble to Python executions? > > Ben > -- Terry Jan Reedy From steve+python at pearwood.info Tue Jun 27 05:57:32 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 27 Jun 2017 19:57:32 +1000 Subject: Syntax error for simple script References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: <59522c0e$0$1617$c3e8da3$5496439d@news.astraweb.com> On Tue, 27 Jun 2017 03:00 am, mm0fmf wrote: > Sorry to victim blame but why can't people copy the error message into a > search engine, such as Google, and see what the problem is themselves? You're not victim blaming. There's no *victim* here, syntax errors are a part of programming and one has to learn to deal with them. That's like a sketch artist whose pencil lead snaps. It happens, and dealing with it is part of the job. We're volunteers, not being paid for our time, and while we're happy to help people with problems the implied social contract goes both ways: people should make an honest effort to solve the problem themselves before asking us to solve it for them. So you're right to ask whether or not the OP tried googling for the error message first. Nothing to apologise for. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ned at nedbatchelder.com Tue Jun 27 06:33:52 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 27 Jun 2017 03:33:52 -0700 (PDT) Subject: "Python launcher" required to run *.py scripts on Windows? In-Reply-To: References: Message-ID: <99665902-4e2a-43d2-b496-efac0c2a3c56@googlegroups.com> On Tuesday, June 27, 2017 at 1:41:22 AM UTC-4, Ben S. wrote: > As I observed v3.6.1 installs (on Windows 7) in addition to the core python engine a second program "Python Launcher". > > As far as I read this component seems to be not necessary since it only aims to facilitate the handling with *.py scripts on Windows. > > When I always call Python script from CommandPrompt like > > D:\tools\Python\python.exe mypythonscript.py > > then this Launcher should not be necessary. > > Am I right? > > Since I prefer to have as less prgrams installed as possible I consider to uninstall the Python Launcher. > > Is this possible without causing trouble to Python executions? Micro-managing installations like this seems like a bad idea. If Python installed it, and you are not running out of disk space, then leave it alone. Perhaps Python will be fine if you remove the launcher, but it's a bad habit to pursue: other software will be harder to untangle, and when problems occur, others will be unable to help because you will now have a completely unique and unsupported configuration. Is there some problem you are trying to solve by removing the launcher? --Ned. From gbs.deadeye at gmail.com Tue Jun 27 06:41:47 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Tue, 27 Jun 2017 10:41:47 +0000 Subject: "Python launcher" required to run *.py scripts on Windows? In-Reply-To: <99665902-4e2a-43d2-b496-efac0c2a3c56@googlegroups.com> References: <99665902-4e2a-43d2-b496-efac0c2a3c56@googlegroups.com> Message-ID: Double Post: https://python-forum.io/Thread-Python-launcher-required-to-run-py-scripts-on-Windows Pleas don't do this. It's not a nice behavior. Thanks. Andre From dhariyalbhaskar at gmail.com Tue Jun 27 06:48:37 2017 From: dhariyalbhaskar at gmail.com (Bhaskar Dhariyal) Date: Tue, 27 Jun 2017 03:48:37 -0700 (PDT) Subject: Converting sentences to vector form Message-ID: <06e7ad25-a1d2-4925-8066-8f644e6b5beb@googlegroups.com> Hi, I am doing a project on data science. I need to convert sentences to vectorial form. I have already written code for cleansing data and already removed stop words and performed stemming. Please help in converting kickdesc and kickkey to vectorial form. Link to code and dataset: https://drive.google.com/open?id=0B1D4AyluMGU0Uk5qYUk3dVAtd00 Apologoes for ugly code Thanks Bhaskar From ben.usenet at bsb.me.uk Tue Jun 27 07:07:44 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 27 Jun 2017 12:07:44 +0100 Subject: Converting sentences to vector form References: <06e7ad25-a1d2-4925-8066-8f644e6b5beb@googlegroups.com> Message-ID: <87vanh1yf3.fsf@bsb.me.uk> Bhaskar Dhariyal writes: > I am doing a project on data science. I need to convert sentences to > vectorial form. That's not a term I am familiar with. If it's standard, that's my fault, but you can still help people like me by explaining what you mean. And if it is not a standard term then people will just be guessing what "vectorial form" is. If it's hard to define you can still help us by showing a few simple examples. Give some sentences, and then give the "vectorial form". > Link to code and dataset: > https://drive.google.com/open?id=0B1D4AyluMGU0Uk5qYUk3dVAtd00 Not everyone is going to follow a link to try to find out what you need. Put as much as you can in the actual posting. > Apologoes for ugly code I didn't find anything there that helped me understand the problem but that may be because I don't have IPython. Maybe loading the ipynb file would show me some code, but it would, again, be better to post the code you are having trouble with here. -- Ben. From dhariyalbhaskar at gmail.com Tue Jun 27 07:15:21 2017 From: dhariyalbhaskar at gmail.com (Bhaskar Dhariyal) Date: Tue, 27 Jun 2017 04:15:21 -0700 (PDT) Subject: Converting sentences to vector form In-Reply-To: <87vanh1yf3.fsf@bsb.me.uk> References: <06e7ad25-a1d2-4925-8066-8f644e6b5beb@googlegroups.com> <87vanh1yf3.fsf@bsb.me.uk> Message-ID: <9207d59d-be07-4ab1-878b-3c37cabebbea@googlegroups.com> You can't train a model on words. You need to convert it into numerical form(vector form). For this there are some packages like word2vec and doc2vec. Thanks for replying On Tuesday, 27 June 2017 16:37:56 UTC+5:30, Ben Bacarisse wrote: > Bhaskar Dhariyal writes: > > > I am doing a project on data science. I need to convert sentences to > > vectorial form. > > That's not a term I am familiar with. If it's standard, that's my > fault, but you can still help people like me by explaining what you > mean. And if it is not a standard term then people will just be > guessing what "vectorial form" is. > > If it's hard to define you can still help us by showing a few simple > examples. Give some sentences, and then give the "vectorial form". > > > > Link to code and dataset: > > https://drive.google.com/open?id=0B1D4AyluMGU0Uk5qYUk3dVAtd00 > > Not everyone is going to follow a link to try to find out what you > need. Put as much as you can in the actual posting. > > > Apologoes for ugly code > > I didn't find anything there that helped me understand the problem but > that may be because I don't have IPython. Maybe loading the ipynb file > would show me some code, but it would, again, be better to post the code > you are having trouble with here. > > -- > Ben. From lynto28 at gmail.com Tue Jun 27 09:55:42 2017 From: lynto28 at gmail.com (Didymus) Date: Tue, 27 Jun 2017 06:55:42 -0700 (PDT) Subject: argparse epilog call function? Message-ID: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> Greetings, I might be barking up the wrong tree, but was wondering if there's a way to have the argpasre epilog call a function. for example: epilog=Examples() Where Examples is: def Examples(): text = """Lots of examples""" print(text.format()) I've place this in and found that it prints out no matter if I use the -h or not and also it prints first.... If I do: epilog='Single Example' it works as intended, unfortunately, I need to show several examples. Just wondering if someone has found a why to do this (without making a custom help). thanks Tom From steve+python at pearwood.info Tue Jun 27 10:05:35 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 28 Jun 2017 00:05:35 +1000 Subject: Syntax error for simple script References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> Message-ID: <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> On Tue, 27 Jun 2017 08:34 am, Erik wrote about the print function error message: py> print x File "", line 1 print x ^ SyntaxError: Missing parentheses in call to 'print' > To be fair, this already seems to be a special case: [...] > >>> len "bar" > File "", line 1 > len "bar" > ^ > SyntaxError: invalid syntax Correct, the print function is singled out as a special case, because people are unlikely to come across tutorials telling them to write len "bar" > If that's the case, then why make it so terse? It's presumably there as > a special case specifically to tell/remind people that print became a > function in Py3. Indeed. But it is hardly terse: it's a full sentence: Missing parentheses in call to 'print' (although missing the full stop at the end, if anyone cares enough to raise a bug report for that). > I think the suggestion above is a bit wordy, but even if it was > something like "Missing parentheses in call to the 'print' function" I think this is a good example of "the curse of knowledge". Its hard for experts to think like a non-expert. It seems obvious to us that adding the word "function" will make it clear to the user what's going on. We understand that print is now a function, and function calls always need parentheses, but that print used to be a statement like while, for, if, etc. that doesn't need parentheses. To a moderately experienced programmer, the word "function" doesn't actually add anything, but it isn't harmful: the error is just a reminder of what they probably already know. But to the beginner, adding "function" at the error might as well be Missing parentheses in call to the 'print' wharrgarbl. It's just jargon. They can either diagnose the problem from the first part of the sentence telling them that there are missing parentheses, or they can't. If they can't, I doubt that adding the word "function" (or any other jargon term like subroutine, procedure or callable) will help. Anyway, that's my opinion. If any newbies, like the OP, would like to venture an opinion, that would be good. That's why I asked Ben if there was something we could do to make the sentence clearer. > or > even "Missing parentheses in call to 'print'. Using 'print' as a > statement is obsolete syntax in Python 3." ... or whatever is considered > more descriptive. When it comes to error messages, more is not always better. There is a balance needed between too terse and too verbose and wordy. Users don't read error messages, even when they are well-written: https://ux.stackexchange.com/questions/393/how-do-i-get-users-to-read-error-messages https://stackoverflow.com/questions/2356698/how-to-get-users-to-read-error-messages so adding more words is not necessarily going to help beginners. In fact it might even make it *less* likely that they read the error. Reading error messages is a skill that needs to be learned, or taught. > Either that or just make it "SyntaxError: invalid syntax" like all the > others. We've been there: adding the specific print message is new to Python 3.5 (or maybe 3.4, I forget which). Python 3.3 gives just "SyntaxError: invalid syntax". -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From firefox at firemail.cc Tue Jun 27 10:09:37 2017 From: firefox at firemail.cc (Fox) Date: Tue, 27 Jun 2017 16:09:37 +0200 Subject: argparse epilog call function? In-Reply-To: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> Message-ID: what " -h " are you even talkin bout ? def Examples(): text = """Lots of examples""" print(text.format()) epilog='where the heck to put a -h ?????? ' epilog=Examples() From rosuav at gmail.com Tue Jun 27 10:11:13 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Jun 2017 00:11:13 +1000 Subject: argparse epilog call function? In-Reply-To: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> Message-ID: On Tue, Jun 27, 2017 at 11:55 PM, Didymus wrote: > Greetings, > > I might be barking up the wrong tree, but was wondering if there's a way to have the argpasre epilog call a function. for example: > > epilog=Examples() > > Where Examples is: > > def Examples(): > text = """Lots of examples""" > print(text.format()) > > I've place this in and found that it prints out no matter if I use the -h or > not and also it prints first.... If I do: > > epilog='Single Example' > > it works as intended, unfortunately, I need to show several examples. Just wondering if someone has found a why to do this (without making a custom help). The way you've written it, Examples() will be called before argparse does its work, and whatever it returns becomes the epilog. Can you work your function such that it returns a string, instead of printing something? Or is it a very expensive function? ChrisA From rosuav at gmail.com Tue Jun 27 10:24:49 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Jun 2017 00:24:49 +1000 Subject: argparse epilog call function? In-Reply-To: References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> Message-ID: On Wed, Jun 28, 2017 at 12:09 AM, Fox wrote: > what " -h " are you even talkin bout ? > > > > > def Examples(): > text = """Lots of examples""" > print(text.format()) > > > > epilog='where the heck to put a -h ?????? ' > > epilog=Examples() List admins, this person has been abusing the pydotorg-www list for several days. Can he please be banned? ChrisA From eryksun at gmail.com Tue Jun 27 10:27:05 2017 From: eryksun at gmail.com (eryk sun) Date: Tue, 27 Jun 2017 14:27:05 +0000 Subject: "Python launcher" required to run *.py scripts on Windows? In-Reply-To: References: Message-ID: On Tue, Jun 27, 2017 at 5:41 AM, Ben S. via Python-list wrote: > > When I always call Python script from CommandPrompt like > > D:\tools\Python\python.exe mypythonscript.py > > then this Launcher should not be necessary. The launcher implements shebang support for directly running Python scripts in multiple versions and virtual environments, including virtual commands (defined in py.ini) to support other implementations such as PyPy. It can also be used to administer multiple CPython installations from the command line. If you don't need it, don't install it. You can install it later if you change your mind. From oitsgp$1hr3$2 at gioia.aioe.org Tue Jun 27 11:10:53 2017 From: oitsgp$1hr3$2 at gioia.aioe.org (Saurabh Chaturvedi) Date: Tue, 27 Jun 2017 15:10:53 +0000 (UTC) Subject: A Good Tutorial on Python Decorators Message-ID: https://opensource.google.com/projects/py-decorators-tutorial From sam_chatsblahbuh at gioia.aioe.org Tue Jun 27 11:23:15 2017 From: sam_chatsblahbuh at gioia.aioe.org (Sam Chats) Date: Tue, 27 Jun 2017 15:23:15 +0000 (UTC) Subject: A million requests per second with Python Message-ID: https://medium.freecodecamp.com/million-requests-per-second-with-python-95c137af319 From __peter__ at web.de Tue Jun 27 11:35:27 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 27 Jun 2017 17:35:27 +0200 Subject: argparse epilog call function? References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> Message-ID: Didymus wrote: > Greetings, > > I might be barking up the wrong tree, but was wondering if there's a way > to have the argpasre epilog call a function. for example: > > epilog=Examples() > > Where Examples is: > > def Examples(): > text = """Lots of examples""" > print(text.format()) > > I've place this in and found that it prints out no matter if I use the -h > or not and also it prints first.... If I do: > > epilog='Single Example' > > it works as intended, unfortunately, I need to show several examples. Just > wondering if someone has found a why to do this (without making a custom > help). Here's a way using a custom *formatter*: $ cat epilog.py import argparse class MyHelpFormatter(argparse.HelpFormatter): def add_text(self, text): if callable(text): text = text() return super().add_text(text) def examples(): return "yadda yadda" parser = argparse.ArgumentParser( epilog=examples, formatter_class=MyHelpFormatter ) parser.add_argument("--foo") parser.parse_args() $ python3 epilog.py -h usage: epilog.py [-h] [--foo FOO] optional arguments: -h, --help show this help message and exit --foo FOO yadda yadda $ From lynto28 at gmail.com Tue Jun 27 11:47:13 2017 From: lynto28 at gmail.com (Didymus) Date: Tue, 27 Jun 2017 08:47:13 -0700 (PDT) Subject: argparse epilog call function? In-Reply-To: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> Message-ID: <6d78f59d-fd7d-4144-b170-1711297a7d0b@googlegroups.com> On Tuesday, June 27, 2017 at 9:56:13 AM UTC-4, Didymus wrote: > Greetings, > > I might be barking up the wrong tree, but was wondering if there's a way to have the argpasre epilog call a function. for example: > > epilog=Examples() > > Where Examples is: > > def Examples(): > text = """Lots of examples""" > print(text.format()) > > I've place this in and found that it prints out no matter if I use the -h or > not and also it prints first.... If I do: > > epilog='Single Example' > > it works as intended, unfortunately, I need to show several examples. Just wondering if someone has found a why to do this (without making a custom help). > > thanks > Tom Well, I thought I had a good idea to work around this by setting a string to the formatted text and then use that string in the epilog: text = Example() # parser = argparse.ArgumentParser(prog=sys.argv[0], description="Example Arg Parse", epilog=text) I had to change the Example function to return the string and in a print it works fine, however when used in the argsparse, the formatting is lost... -T From lynto28 at gmail.com Tue Jun 27 12:20:55 2017 From: lynto28 at gmail.com (Didymus) Date: Tue, 27 Jun 2017 09:20:55 -0700 (PDT) Subject: argparse epilog call function? In-Reply-To: <6d78f59d-fd7d-4144-b170-1711297a7d0b@googlegroups.com> References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> <6d78f59d-fd7d-4144-b170-1711297a7d0b@googlegroups.com> Message-ID: <5a66c164-beac-470e-9bec-779d2abce9e0@googlegroups.com> On Tuesday, June 27, 2017 at 11:47:42 AM UTC-4, Didymus wrote: > On Tuesday, June 27, 2017 at 9:56:13 AM UTC-4, Didymus wrote: > > Greetings, > > > > I might be barking up the wrong tree, but was wondering if there's a way to have the argpasre epilog call a function. for example: > > > > epilog=Examples() > > > > Where Examples is: > > > > def Examples(): > > text = """Lots of examples""" > > print(text.format()) > > > > I've place this in and found that it prints out no matter if I use the -h or > > not and also it prints first.... If I do: > > > > epilog='Single Example' > > > > it works as intended, unfortunately, I need to show several examples. Just wondering if someone has found a why to do this (without making a custom help). > > > > thanks > > Tom > Well, I thought I had a good idea to work around this by setting a string to the formatted text and then use that string in the epilog: > > text = Example() > # > parser = argparse.ArgumentParser(prog=sys.argv[0], > description="Example Arg Parse", > epilog=text) > > I had to change the Example function to return the string and in a print it works fine, however when used in the argsparse, the formatting is lost... > > -T Greetings, I got what I was looking for: def Examples(): text = """Lots of examples""" return text.format() parser = argparse.ArgumentParser(prog=sys.argv[0], formatter_class=argparse.RawTextHelpFormatter, description="Argparse Example", epilog=text) The key here is the "formatter_class=argparse.RawTestHelpFormatter". Once I set that the epilog prints out nicely formatted: % ./test.py -h usage: ./test.py [-h] [-n NAME] [-f | -m | -r | -v] Argparse Example. optional arguments: -h, --help show this help message and exit -n NAME, --name NAME Name to be created. Examples: Ex 1: ./test.py -n juser Ex 2: ./test.py -n juser -r ... Thanks for all the help. -Tom From sam_chatsblahbuh at gioia.aioe.org Tue Jun 27 12:23:41 2017 From: sam_chatsblahbuh at gioia.aioe.org (Sam Chats) Date: Tue, 27 Jun 2017 16:23:41 +0000 (UTC) Subject: How to build a simple neural network in 9 lines of Python code Message-ID: https://medium.com/technology-invention-and-more/how-to-build-a-simple-neural-network-in-9-lines-of-python-code-cc8f23647ca1 From pkpearson at nowhere.invalid Tue Jun 27 12:33:42 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 27 Jun 2017 16:33:42 GMT Subject: A Good Tutorial on Python Decorators References: Message-ID: On Tue, 27 Jun 2017 15:10:53 +0000 (UTC), Saurabh Chaturvedi wrote: > https://opensource.google.com/projects/py-decorators-tutorial "No Results found." -- To email me, substitute nowhere->runbox, invalid->com. From gbs.deadeye at gmail.com Tue Jun 27 12:43:38 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Tue, 27 Jun 2017 16:43:38 +0000 Subject: A Good Tutorial on Python Decorators In-Reply-To: References: Message-ID: Activate JavaScript, then you can see the content. I had the same problem. Peter Pearson schrieb am Di., 27. Juni 2017 um 18:35 Uhr: > On Tue, 27 Jun 2017 15:10:53 +0000 (UTC), Saurabh Chaturvedi wrote: > > https://opensource.google.com/projects/py-decorators-tutorial > > "No Results found." > > -- > To email me, substitute nowhere->runbox, invalid->com. > -- > https://mail.python.org/mailman/listinfo/python-list > From robertvstepp at gmail.com Tue Jun 27 12:52:29 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Tue, 27 Jun 2017 11:52:29 -0500 Subject: Syntax error for simple script In-Reply-To: <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Jun 27, 2017 at 9:05 AM, Steve D'Aprano wrote: > On Tue, 27 Jun 2017 08:34 am, Erik wrote about the print function error message: > > py> print x > File "", line 1 > print x > ^ > SyntaxError: Missing parentheses in call to 'print' [snip] >> I think the suggestion above is a bit wordy, but even if it was >> something like "Missing parentheses in call to the 'print' function" > > I think this is a good example of "the curse of knowledge". Its hard for experts > to think like a non-expert. > > It seems obvious to us that adding the word "function" will make it clear to the > user what's going on. We understand that print is now a function, and function > calls always need parentheses, but that print used to be a statement like > while, for, if, etc. that doesn't need parentheses. > > To a moderately experienced programmer, the word "function" doesn't actually add > anything, but it isn't harmful: the error is just a reminder of what they > probably already know. > > But to the beginner, adding "function" at the error might as well be > > Missing parentheses in call to the 'print' wharrgarbl. > > > It's just jargon. They can either diagnose the problem from the first part of > the sentence telling them that there are missing parentheses, or they can't. If > they can't, I doubt that adding the word "function" (or any other jargon term > like subroutine, procedure or callable) will help. > > Anyway, that's my opinion. If any newbies, like the OP, would like to venture an > opinion, that would be good. That's why I asked Ben if there was something we > could do to make the sentence clearer. Perhaps add a simple usage example? SyntaxError: Missing parentheses in call to 'print'. Correct example: print('Prints this string') Surely a little more verbiage won't be too harmful? -- boB From marko at pacujo.net Tue Jun 27 12:57:05 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 27 Jun 2017 19:57:05 +0300 Subject: How to build a simple neural network in 9 lines of Python code References: Message-ID: <871sq5z7vi.fsf@elektro.pacujo.net> Sam Chats : > https://medium.com/technology-invention-and-more/how-to-build-a-simpl > e-neural-network-in-9-lines-of-python-code-cc8f23647ca1 Impressive, but APL's got Python beat. This one-liner implements the Game of Life: life?{?1 ??.?3 4=+/,?1 0 1?.??1 0 1?.???} "Donc Dieu existe, r?pondez !" Marko From paul.james.barry at gmail.com Tue Jun 27 13:20:42 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Tue, 27 Jun 2017 18:20:42 +0100 Subject: A Good Tutorial on Python Decorators In-Reply-To: References: Message-ID: Me doing the decorator thing (from a web dev point-of-view) at PyCon Ireland 2015: https://www.youtube.com/watch?v=My2UpCaN7rE and here's a link to the PDF of my talk's material: http://paulbarry.itcarlow.ie/pyconie2015/decorators.pdf Paul. On 27 June 2017 at 17:43, Andre M?ller wrote: > Activate JavaScript, then you can see the content. > I had the same problem. > > Peter Pearson schrieb am Di., 27. Juni 2017 um > 18:35 Uhr: > > > On Tue, 27 Jun 2017 15:10:53 +0000 (UTC), Saurabh Chaturvedi wrote: > > > https://opensource.google.com/projects/py-decorators-tutorial > > > > "No Results found." > > > > -- > > To email me, substitute nowhere->runbox, invalid->com. > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From breamoreboy at gmail.com Tue Jun 27 14:03:59 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Tue, 27 Jun 2017 11:03:59 -0700 (PDT) Subject: argparse epilog call function? In-Reply-To: References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> Message-ID: <8354d529-907d-4b69-80b3-a2ebcda15698@googlegroups.com> On Tuesday, June 27, 2017 at 3:25:10 PM UTC+1, Chris Angelico wrote: > On Wed, Jun 28, 2017 at 12:09 AM, Fox wrote: > > what " -h " are you even talkin bout ? > > > > > > > > > > def Examples(): > > text = """Lots of examples""" > > print(text.format()) > > > > > > > > epilog='where the heck to put a -h ?????? ' > > > > epilog=Examples() > > List admins, this person has been abusing the pydotorg-www list for > several days. Can he please be banned? > > ChrisA I'll second that, the wxpython lists have been suffering similar abuse for several days. Kindest regards. Mark Lawrence. From john_ladasky at sbcglobal.net Tue Jun 27 15:31:19 2017 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Tue, 27 Jun 2017 12:31:19 -0700 (PDT) Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: References: Message-ID: On Tuesday, June 27, 2017 at 9:24:07 AM UTC-7, Sam Chats wrote: > https://medium.com/technology-invention-and-more/how-to-build-a-simple-neural-network-in-9-lines-of-python-code-cc8f23647ca1 OK, that's cheating a bit, using Numpy. It's a nice little program, but it leverages a huge, powerful library. From marko at pacujo.net Tue Jun 27 15:34:18 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 27 Jun 2017 22:34:18 +0300 Subject: How to build a simple neural network in 9 lines of Python code References: Message-ID: <87r2y5xm11.fsf@elektro.pacujo.net> John Ladasky : > OK, that's cheating a bit, using Numpy. It's a nice little program, > but it leverages a huge, powerful library. What would *not* be cheating? A language without a library would be dead. Marko From sanscsantosh at gmail.com Tue Jun 27 15:36:26 2017 From: sanscsantosh at gmail.com (SANS SANTOSH) Date: Tue, 27 Jun 2017 19:36:26 +0000 Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency In-Reply-To: References: Message-ID: On Sat 24. Jun 2017 at 13:27, Mark Byrne wrote: > A problem (possibly the problem) is the lines which use the get function: > count = frequency.get(word,0) > > Since the dictionary is empty at the start of the loop, the get function is > passing a value of 0 to count and count1. > The subsequent updates to the dictionary are applying a value of zero > > As a result, the file being written to contains count values of 0 for each > word. > > Possible fix is to replace this: > > count = frequency.get(word,0) > count1 = frequency.get(word1,0) > if word1 == word: > frequency[word] = count + count1 > else: > frequency[word] = count > > with this: > > if word1 == word: > if word in frequency: > frequency[word] += 1 > else: > frequency[word] = 1 > -- > https://mail.python.org/mailman/listinfo/python-list > From python at lucidity.plus.com Tue Jun 27 15:38:46 2017 From: python at lucidity.plus.com (Erik) Date: Tue, 27 Jun 2017 20:38:46 +0100 Subject: Syntax error for simple script In-Reply-To: <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5134b7c7-f97b-d1a0-2743-4af18964e8c2@lucidity.plus.com> On 27/06/17 15:05, Steve D'Aprano wrote: > On Tue, 27 Jun 2017 08:34 am, Erik wrote about the print function error message: >> To be fair, this already seems to be a special case: > [...] >> >>> len "bar" >> File "", line 1 >> len "bar" >> ^ >> SyntaxError: invalid syntax > > Correct, the print function is singled out as a special case, because people are > unlikely to come across tutorials telling them to write > > len "bar" Sure. My point was that it's obviously a special case and not a generic error message that would occur in the same situation with any other callables. Hence there is obviously a single, specific place in the parser code that could just be changed slightly if it would help. I haven't looked at the source code for this. > Indeed. But it is hardly terse: it's a full sentence: > > Missing parentheses in call to 'print' Right, but as in this case, it's likely to trip up a beginner who happens to be following a Py2 tutorial or copying Py2 examples from SO or somewhere. In fact, apart from those of us who switch between versions for different projects and just forget (old habits die hard), a beginner typing in a Py2 example is probably the only case where this error is seen. So I don't understand the resistance to making the error be a bit more explicit about why it is being generated to help those users - it doesn't explain (the obvious reason) _why_ their tutorial/example gives them an error when it doesn't to the person who wrote the tutorial. Note that I am *explicitly* making Py3 the "good guy" and Py2 the "bad guy" in all the wordings that I have given that mention a version number - it could even be something like "Missing parentheses in call to 'print'. This is obsolete Python 2 syntax. Please use modern Python 3 syntax.". This addresses ChrisA's issue with the proposed wordy error that he thought Py3 was playing second fiddle in that version. One other possibility is that a beginner fires up Py3 and gets that "weird" (to them) error when they tried their tutorial or SO code and then finds out that if they fire up Py2 instead it just works. Now they might continue to use Py2 because it's "the one that works" for them. I would suggest that making the error dissuade them from doing that (for the cost of a few tens of bytes) is a good thing. >> I think the suggestion above is a bit wordy, but even if it was >> something like "Missing parentheses in call to the 'print' function" > > I think this is a good example of "the curse of knowledge". Its hard for expert > to think like a non-expert. [snip] > But to the beginner, adding "function" at the error might as well be > > Missing parentheses in call to the 'print' wharrgarbl. You are concentrating on the detail of what I wrote. I don't care what the eventual wording is (and for sure, someone else is far more qualified than me to suggest wording that is better for a beginner). My point is simply that the error _could_ contain _some_ more information that addresses this issue for a beginner who may be typing in Py2 examples that encourages them to seek out Py3 tutorials instead. > That's why I asked Ben if there was something we > could do to make the sentence clearer. Exactly. Ben is an example of someone more qualified than me to suggest the correct words. >> Either that or just make it "SyntaxError: invalid syntax" like all the >> others. > > We've been there: adding the specific print message is new to Python 3.5 (or > maybe 3.4, I forget which). Python 3.3 gives just "SyntaxError: invalid > syntax". That surprises me, to be honest. I had presumed that the parser had just had the machinery for that message placed where the 'print' keyword used to be handled rather than ripping the whole thing out when it became a function. Interesting that it's a retrospective special case. E. From alister.ware at ntlworld.com Tue Jun 27 15:42:30 2017 From: alister.ware at ntlworld.com (alister) Date: Tue, 27 Jun 2017 19:42:30 GMT Subject: How to build a simple neural network in 9 lines of Python code References: <87r2y5xm11.fsf@elektro.pacujo.net> Message-ID: On Tue, 27 Jun 2017 22:34:18 +0300, Marko Rauhamaa wrote: > John Ladasky : >> OK, that's cheating a bit, using Numpy. It's a nice little program, >> but it leverages a huge, powerful library. > > What would *not* be cheating? A language without a library would be > dead. > > > Marko true but for a realistic comparison you should probably count all the lines of code in the libruary -- I cannot believe that God plays dice with the cosmos. -- Albert Einstein, on the randomness of quantum mechanics From rosuav at gmail.com Tue Jun 27 15:59:48 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Jun 2017 05:59:48 +1000 Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: <87r2y5xm11.fsf@elektro.pacujo.net> References: <87r2y5xm11.fsf@elektro.pacujo.net> Message-ID: On Wed, Jun 28, 2017 at 5:34 AM, Marko Rauhamaa wrote: > John Ladasky : >> OK, that's cheating a bit, using Numpy. It's a nice little program, >> but it leverages a huge, powerful library. > > What would *not* be cheating? A language without a library would be > dead. Sure, but there are different levels of cheating. Using a general-purpose programming language and its standard library isn't usually considered cheating, but using a language or library that's specifically designed for this purpose is less about "hey look how simple this is" and more about "hey look how awesome this lang/lib is". Which is a perfectly reasonable thing to brag; Python is beautifully expressive in the general case, but there are some amazing tools for special purposes. So I wouldn't call it cheating; it's a demonstration of the expressiveness of numpy. ChrisA From malaclypse2 at gmail.com Tue Jun 27 16:06:49 2017 From: malaclypse2 at gmail.com (Jerry Hill) Date: Tue, 27 Jun 2017 16:06:49 -0400 Subject: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency In-Reply-To: References: Message-ID: On Fri, Jun 23, 2017 at 4:07 PM, Mark Byrne wrote: > Possible fix is to replace this: > > count = frequency.get(word,0) > count1 = frequency.get(word1,0) > if word1 == word: > frequency[word] = count + count1 > else: > frequency[word] = count > > with this: > > if word1 == word: > if word in frequency: > frequency[word] += 1 > else: > frequency[word] = 1 > Have you considered replacing your frequency dict with a defaultdict(int)?? That way you could boil the whole thing down to: from collections import defaultdict frequency = defaultdict(int) ... ... if word1 == word: frequency[word] += 1 The defaultdict takes care of the special case for you. If the value is missing, it's defaulted to 0. -- Jerry From marko at pacujo.net Tue Jun 27 16:19:09 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 27 Jun 2017 23:19:09 +0300 Subject: How to build a simple neural network in 9 lines of Python code References: <87r2y5xm11.fsf@elektro.pacujo.net> Message-ID: <87mv8txjya.fsf@elektro.pacujo.net> alister : > On Tue, 27 Jun 2017 22:34:18 +0300, Marko Rauhamaa wrote: > >> John Ladasky : >>> OK, that's cheating a bit, using Numpy. It's a nice little program, >>> but it leverages a huge, powerful library. >> >> What would *not* be cheating? A language without a library would be >> dead. > > true but for a realistic comparison you should probably count all the > lines of code in the libruary How's that defined? Lines of Python? Lines of C? Lines of assembly? Lines of microcode? What is a line? Marko From marko at pacujo.net Tue Jun 27 16:22:36 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 27 Jun 2017 23:22:36 +0300 Subject: How to build a simple neural network in 9 lines of Python code References: <87r2y5xm11.fsf@elektro.pacujo.net> Message-ID: <87injhxjsj.fsf@elektro.pacujo.net> Chris Angelico : > On Wed, Jun 28, 2017 at 5:34 AM, Marko Rauhamaa wrote: >> What would *not* be cheating? A language without a library would be >> dead. > > Sure, but there are different levels of cheating. Using a > general-purpose programming language and its standard library isn't > usually considered cheating, but using a language or library that's > specifically designed for this purpose is less about "hey look how > simple this is" and more about "hey look how awesome this lang/lib > is". Which is a perfectly reasonable thing to brag; Python is > beautifully expressive in the general case, but there are some amazing > tools for special purposes. > > So I wouldn't call it cheating; it's a demonstration of the > expressiveness of numpy. You saw the APL example, right? APL's standard runtime/library contains most of Numpy functionality because that's what APL has been designed for. Is that cheating? Marko From rosuav at gmail.com Tue Jun 27 16:52:51 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Jun 2017 06:52:51 +1000 Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: <87injhxjsj.fsf@elektro.pacujo.net> References: <87r2y5xm11.fsf@elektro.pacujo.net> <87injhxjsj.fsf@elektro.pacujo.net> Message-ID: On Wed, Jun 28, 2017 at 6:22 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Wed, Jun 28, 2017 at 5:34 AM, Marko Rauhamaa wrote: >>> What would *not* be cheating? A language without a library would be >>> dead. >> >> Sure, but there are different levels of cheating. Using a >> general-purpose programming language and its standard library isn't >> usually considered cheating, but using a language or library that's >> specifically designed for this purpose is less about "hey look how >> simple this is" and more about "hey look how awesome this lang/lib >> is". Which is a perfectly reasonable thing to brag; Python is >> beautifully expressive in the general case, but there are some amazing >> tools for special purposes. >> >> So I wouldn't call it cheating; it's a demonstration of the >> expressiveness of numpy. > > You saw the APL example, right? APL's standard runtime/library contains > most of Numpy functionality because that's what APL has been designed > for. > > Is that cheating? No, for the same reason. What it demonstrates is the terseness of APL when used for its exact use case. Ultimately, everything is implemented using lower level code. Isn't it impressive how easily a computer can add two integers? Trying to do the same thing in dominoes takes a lot of code: http://www.numberphile.com/videos/domino_circuit.html Should we count program code in domino equivalence? ChrisA From blahBlah at blah.org Tue Jun 27 17:38:45 2017 From: blahBlah at blah.org (YOUR_NAME_HERE) Date: Tue, 27 Jun 2017 21:38:45 +0000 (UTC) Subject: Check A million requests per second with Python Message-ID: Awesome! Python's becoming more of an all rounder! From blahBlah at blah.org Tue Jun 27 18:22:41 2017 From: blahBlah at blah.org (YOUR_NAME_HERE) Date: Tue, 27 Jun 2017 22:22:41 +0000 (UTC) Subject: A million requests per second with Python References: Message-ID: Awesome! I love Python! From steve+python at pearwood.info Tue Jun 27 21:14:59 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 28 Jun 2017 11:14:59 +1000 Subject: How to build a simple neural network in 9 lines of Python code References: Message-ID: <59530314$0$1610$c3e8da3$5496439d@news.astraweb.com> On Wed, 28 Jun 2017 02:23 am, Sam Chats wrote: > https://medium.com/technology-invention-and-more/how-to-build-a-simple-neural-network-in-9-lines-of-python-code-cc8f23647ca1 The derivative of the sigmoid curve given is completely wrong. def __sigmoid(self, x): return 1 / (1 + exp(-x)) def __sigmoid_derivative(self, x): return x * (1 - x) Should be: def __sigmoid_derivative(self, x): return exp(x) / (1 + exp(x))**2 http://mathworld.wolfram.com/SigmoidFunction.html I wish these neural networks would give a tutorial on how to do something non-trivial where: - your training data is not just a couple of bits; - the function you want the neural network to perform is non-trivial. E.g. I have a bunch of data: ['cheese', 'pirate', 'aardvark', 'vitamin', 'egg', 'moon'] and a set of flags: [True, False, True, True, False, True] and I want the network to predict that: 'calcium' -> False 'aluminium' -> True 'wood' -> True 'plastic' -> False (The function is, "is there a duplicated vowel?") -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Jun 27 21:20:47 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 28 Jun 2017 11:20:47 +1000 Subject: How to build a simple neural network in 9 lines of Python code References: <87r2y5xm11.fsf@elektro.pacujo.net> Message-ID: <59530470$0$1615$c3e8da3$5496439d@news.astraweb.com> On Wed, 28 Jun 2017 05:34 am, Marko Rauhamaa wrote: > John Ladasky : >> OK, that's cheating a bit, using Numpy. It's a nice little program, >> but it leverages a huge, powerful library. > > What would *not* be cheating? A language without a library would be > dead. Its not really nine lines of Python. It's nine lines of Python, to call a library of many thousands of lines of C, Fortran and Python. And its not really a neural *network* as such, more like just a single neuron. Starting from scratch with just the Python language and standard library, no external third party code, how quickly can you write a neural network? I think the answer is probably a wee bit more than "nine lines". -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Jun 27 21:24:03 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 28 Jun 2017 11:24:03 +1000 Subject: How to build a simple neural network in 9 lines of Python code References: <87r2y5xm11.fsf@elektro.pacujo.net> <87mv8txjya.fsf@elektro.pacujo.net> Message-ID: <59530534$0$1615$c3e8da3$5496439d@news.astraweb.com> On Wed, 28 Jun 2017 06:19 am, Marko Rauhamaa wrote: > alister : > >> On Tue, 27 Jun 2017 22:34:18 +0300, Marko Rauhamaa wrote: >> >>> John Ladasky : >>>> OK, that's cheating a bit, using Numpy. It's a nice little program, >>>> but it leverages a huge, powerful library. >>> >>> What would *not* be cheating? A language without a library would be >>> dead. >> >> true but for a realistic comparison you should probably count all the >> lines of code in the libruary > > How's that defined? Lines of Python? Lines of C? Lines of assembly? > Lines of microcode? > > What is a line? Microcode isn't text so it doesn't come in lines. Obviously we're talking about source code, not object code, byte code or machine code, since source code is the only one that is measured in lines rather than bytes. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Jun 27 22:28:25 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 28 Jun 2017 12:28:25 +1000 Subject: How to build a simple neural network in 9 lines of Python code References: <87r2y5xm11.fsf@elektro.pacujo.net> <87injhxjsj.fsf@elektro.pacujo.net> Message-ID: <5953144a$0$22140$c3e8da3$5496439d@news.astraweb.com> On Wed, 28 Jun 2017 06:22 am, Marko Rauhamaa wrote: > You saw the APL example, right? APL's standard runtime/library contains > most of Numpy functionality because that's what APL has been designed > for. > > Is that cheating? Of course not. That demonstrates beautifully (or perhaps "unreadably tersely") that the APL language primitives are extremely powerful (too powerful?). Apart from just your usual contrariness *wink* I don't know why you are objecting to this. It should be obvious that if you allow the use of external libraries that can contain arbitrary amounts of code, *without* counting that external code towards your measure of code complexity, you get a bogus measurement of code complexity. (That's like manufacturers who can make things artificially cheaply because they don't have to pay the full costs of their input and processes: they get their raw materials subsidised, putting part of the cost on tax payers, and don't have to pay for their pollution, making others pay the cost.) Suppose I said that I can write a full featured, advanced GUI web browser, complete with Javascript and a plug-in system, in just *two* lines of Python code: import webbrowser webbrowser.open('http://www.example.com') What an astonishingly powerful language Python must be! Other languages require dozens of lines of code just to parse Javascript, let alone run it. Except... I'm not really measuring the capability of *Python*, as such. Not if I word it as I have. I haven't really implemented a web browser, I'm just using an external web browser. If I wanted to change the parameters, lets say by changing the Javascript keyword "true" to "truish", then my code would expand from two lines to millions of lines. My demo of launching an external process to do all the work is of no help at all to assist a developer in estimating the expressiveness and power of the language. If I were more honest, I would say: "Here is how I can launch a web browser using the Python standard library in just two lines of code." which is an honest description of what I did, and can allow people to make fair comparisons. E.g. comparing Python to AcmeScript, where you write: program myprogram begin program load library webbrowser new string url copied from 'http://www.example.com' with webbrowser begin method = open method.call url end end program (I made that language up, for the record, don't bother googling for it.) That gives you a fair comparison of the language expressiveness and power. Chris is right: saying that we can create a simple neural network in 9 lines of Python plus numpy allows fair comparison to other combinations of language and library. Implying that it's just Python alone does not. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Jun 27 22:46:57 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 28 Jun 2017 12:46:57 +1000 Subject: Syntax error for simple script References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> <5134b7c7-f97b-d1a0-2743-4af18964e8c2@lucidity.plus.com> Message-ID: <595318a3$0$1602$c3e8da3$5496439d@news.astraweb.com> On Wed, 28 Jun 2017 05:38 am, Erik wrote: [...] > So I don't understand the resistance to making the error be a bit more > explicit about why it is being generated to help those users - it > doesn't explain (the obvious reason) _why_ their tutorial/example gives > them an error when it doesn't to the person who wrote the tutorial. All of this is irrelevant if beginners don't read the error message. End-users don't read error messages, no matter how well written they are, and every word you add probably cuts the number of people reading it by half. Newbie programmers are more like end-users than seasoned programmers. Learning to read the error message is a skill that must be taught and learned. Part of the learning process is to explain the why, the when, the how, and perhaps even the history of the error. That's not something needed in the error message itself (imagine how annoying it would be if ZeroDivisionError tried to explain the basis of arithmetic and why you can't divide by zero), and rather than helping the newbie, it will probably intimidate them and make it even less likely to read and understand the error. When designing an error message, imagine that you are going to read it a dozen times a day, every day, for the rest of your career. > One other possibility is that a beginner fires up Py3 and gets that > "weird" (to them) error when they tried their tutorial or SO code and > then finds out that if they fire up Py2 instead it just works. Now they > might continue to use Py2 because it's "the one that works" for them. > > I would suggest that making the error dissuade them from doing that (for > the cost of a few tens of bytes) is a good thing. The cost isn't ten bytes. The cost is human attention span. Don't think of the newbie reading the error for the first time, because that only happens *once*. Think of them reading it for the 50th time. Do you think they will still appreciate your little history lesson? > You are concentrating on the detail of what I wrote. I don't care what > the eventual wording is (and for sure, someone else is far more > qualified than me to suggest wording that is better for a beginner). My > point is simply that the error _could_ contain _some_ more information > that addresses this issue for a beginner who may be typing in Py2 > examples that encourages them to seek out Py3 tutorials instead. Of course it could. It could also include a link to the Python documentation, a history of how and why Python 3000 came to be, and a note that if you email the Python Software Foundation they'll pay you $50, and none of these things will make the slightest difference if people don't read them. It's not a question of what the error message *could* say, its what it *should* say, and more is not always better. I think that there are broadly two sets of newbies in the world: - those who will read the message, and be able to resolve the problem from the current wording ("oh, it needs parentheses, like a function"); - and those who won't, in which case adding more words to the message won't help in the slightest. >> That's why I asked Ben if there was something we >> could do to make the sentence clearer. > > Exactly. Ben is an example of someone more qualified than me to suggest > the correct words. And is typical, Ben has not (as far as I can see) replied. Which I take as an answer to my question: no, he did not read the error message, or spend more than a millisecond trying to understand it, and adding more words wouldn't have changed that in the slightest. "Cynical" is what Pollyannas call realists :-) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From flebber.crue at gmail.com Tue Jun 27 22:48:07 2017 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Tue, 27 Jun 2017 19:48:07 -0700 (PDT) Subject: Create a list of dates for same day of week in a year Message-ID: <08dc0aab-d7ea-41e5-bc05-60e3c05c62f0@googlegroups.com> Afternoon Is there an obvious method I am missing in creating a list of dates? I want to get a list of each Saturday and each Wednesday for the year 2017. It seems and maybe this is where I am wrong but doesn't the datetime library already know the dates if yes is there an easy way to query it? Also checked out Python Arrow http://arrow.readthedocs.io/en/latest/ as it expands on the python standard library but I couldn't find an obvious example of this. Thoughts or examples? Cheers Sayth From rosuav at gmail.com Tue Jun 27 23:07:42 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Jun 2017 13:07:42 +1000 Subject: Create a list of dates for same day of week in a year In-Reply-To: <08dc0aab-d7ea-41e5-bc05-60e3c05c62f0@googlegroups.com> References: <08dc0aab-d7ea-41e5-bc05-60e3c05c62f0@googlegroups.com> Message-ID: On Wed, Jun 28, 2017 at 12:48 PM, Sayth Renshaw wrote: > Is there an obvious method I am missing in creating a list of dates? I want to get a list of each Saturday and each Wednesday for the year 2017. > > It seems and maybe this is where I am wrong but doesn't the datetime library already know the dates if yes is there an easy way to query it? > Sorta-kinda. >>> import datetime >>> today = datetime.date.today() >>> monday = today - datetime.timedelta(days=today.weekday()) >>> wednesday = monday + datetime.timedelta(days=2) >>> saturday = monday + datetime.timedelta(days=5) >>> week = datetime.timedelta(days=7) >>> next_wed = wednesday + week You can mess around with that. If you want to find the beginning of the year, you could start by using datetime.date(2017, 1, 1) instead of today(), and then locate the previous Monday the same way. So yes, all the information is available, but no, there's no easy way to say "give me all the Saturdays in 2017". You'd have to iterate. ChrisA From ian.g.kelly at gmail.com Tue Jun 27 23:17:44 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 27 Jun 2017 21:17:44 -0600 Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: <59530314$0$1610$c3e8da3$5496439d@news.astraweb.com> References: <59530314$0$1610$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tuesday, June 27, 2017, Steve D'Aprano wrote: > On Wed, 28 Jun 2017 02:23 am, Sam Chats wrote: > > > > https://medium.com/technology-invention-and-more/how-to- > build-a-simple-neural-network-in-9-lines-of-python-code-cc8f23647ca1 > > > The derivative of the sigmoid curve given is completely wrong. > > def __sigmoid(self, x): > return 1 / (1 + exp(-x)) > > def __sigmoid_derivative(self, x): > return x * (1 - x) > > > Should be: > > def __sigmoid_derivative(self, x): > return exp(x) / (1 + exp(x))**2 > > > > http://mathworld.wolfram.com/SigmoidFunction.html > > Actually, it's not stated clearly but x is not the same in each function. The return value of sigmoid is the input of sigmoid_derivative. The article that you linked confirms that this identity holds. From rosuav at gmail.com Tue Jun 27 23:19:24 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Jun 2017 13:19:24 +1000 Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: <5953144a$0$22140$c3e8da3$5496439d@news.astraweb.com> References: <87r2y5xm11.fsf@elektro.pacujo.net> <87injhxjsj.fsf@elektro.pacujo.net> <5953144a$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Jun 28, 2017 at 12:28 PM, Steve D'Aprano wrote: > E.g. comparing Python to AcmeScript, where you write: > > > program myprogram > begin program > load library webbrowser > new string url copied from 'http://www.example.com' > with webbrowser > begin > method = open > method.call url > end > end program > > (I made that language up, for the record, don't bother googling for it.) Just to be contrary, I did. And Google suggested that maybe I meant "acnescript". Hmmmmmmmmmmmm. ChrisA From rosuav at gmail.com Tue Jun 27 23:30:33 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Jun 2017 13:30:33 +1000 Subject: Syntax error for simple script In-Reply-To: <595318a3$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> <5134b7c7-f97b-d1a0-2743-4af18964e8c2@lucidity.plus.com> <595318a3$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Jun 28, 2017 at 12:46 PM, Steve D'Aprano wrote: > I think that there are broadly two sets of newbies in the world: > > - those who will read the message, and be able to resolve the problem from the > current wording ("oh, it needs parentheses, like a function"); > > - and those who won't, in which case adding more words to the message won't help > in the slightest. And in between are the people who read the message but don't understand it, and then key it into a web search engine. For those people, the value isn't necessarily in the clarity of the wording, but in the specificity of it; as long as Google/DuckDuckGo/Bing/Yahoo can find the right information given that error as input, it's good enough. As such, I think the current wording is excellent; I start typing "missing parentheses" and all four of those search engines suggest the rest of the message, and all of them have good results (mainly from Stack Overflow) at the top of the search. Bing and DDG even pick up part of the answer to show above the results, with DDG edging out Bing in usefulness. Or maybe they're a subcategory of the first group. I don't know. In any case, consistency helps the search engines (it's easier to search for "missing parentheses in call to print" than for "unexpected error at <0x13456474678fbcea>" where the hex number varies according to cause), and brevity (as long as clarity isn't sacrificed) is a huge advantage in getting the message typed in. ChrisA From rustompmody at gmail.com Tue Jun 27 23:56:24 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 27 Jun 2017 20:56:24 -0700 (PDT) Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: <87r2y5xm11.fsf@elektro.pacujo.net> References: <87r2y5xm11.fsf@elektro.pacujo.net> Message-ID: On Wednesday, June 28, 2017 at 1:04:46 AM UTC+5:30, Marko Rauhamaa wrote: > John Ladasky > > OK, that's cheating a bit, using Numpy. It's a nice little program, > > but it leverages a huge, powerful library. > > What would *not* be cheating? A language without a library would be > dead. One man's powerful is another's simple A beginning C programmer treats lists as a "data structure" for an advanced course on data structures A beginning python programmer starts using them in his first hour or at worst day Likewise python does this with the help of numpy >From which one could conclude one of - This (numpy-using) 9 lines is advanced - Numpy needs to be in python's core My guess of Marko's intent of bringing up APL is just to show a language in which numpy is effectively in the language core My other guess is that these same 9 lines could be translated to R with no import PS I am not advocating numpybuiltintopython From flebber.crue at gmail.com Wed Jun 28 00:15:21 2017 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Tue, 27 Jun 2017 21:15:21 -0700 (PDT) Subject: Create a list of dates for same day of week in a year In-Reply-To: References: <08dc0aab-d7ea-41e5-bc05-60e3c05c62f0@googlegroups.com> Message-ID: > > Is there an obvious method I am missing in creating a list of dates? I want to get a list of each Saturday and each Wednesday for the year 2017. > > > > It seems and maybe this is where I am wrong but doesn't the datetime library already know the dates if yes is there an easy way to query it? > > > > Sorta-kinda. > > >>> import datetime > >>> today = datetime.date.today() > >>> monday = today - datetime.timedelta(days=today.weekday()) > >>> wednesday = monday + datetime.timedelta(days=2) > >>> saturday = monday + datetime.timedelta(days=5) > >>> week = datetime.timedelta(days=7) > >>> next_wed = wednesday + week > > You can mess around with that. If you want to find the beginning of > the year, you could start by using datetime.date(2017, 1, 1) instead > of today(), and then locate the previous Monday the same way. > > So yes, all the information is available, but no, there's no easy way > to say "give me all the Saturdays in 2017". You'd have to iterate. > > ChrisA Thanks ChrisA could using the calendar and a for loop work better? As below test a date to see if the value = 5 so loop months 1-12 and then days 1-31 and just throw an error when the date doesn't exist 30th of Feb etc In [19]: import calendar In [23]: calendar.weekday(2017,6,24) Out[23]: 5 Or I have set the Saturday to be the firstday of the week https://docs.python.org/2/library/calendar.html is there a way to loop for all firstdayofweeks in year and return dates out that way? Not sure how to get all dates in the year as an object for iteration though. The calendar iter items end at months https://docs.python.org/2/library/calendar.html#calendar.Calendar.itermonthdays In [20]: calendar.setfirstweekday(calendar.SATURDAY) In [21]: calendar.firstweekday() Out[21]: 5 Cheers Sayth From waldemar.osuch at gmail.com Wed Jun 28 02:00:45 2017 From: waldemar.osuch at gmail.com (waldemar.osuch at gmail.com) Date: Tue, 27 Jun 2017 23:00:45 -0700 (PDT) Subject: Create a list of dates for same day of week in a year In-Reply-To: <08dc0aab-d7ea-41e5-bc05-60e3c05c62f0@googlegroups.com> References: <08dc0aab-d7ea-41e5-bc05-60e3c05c62f0@googlegroups.com> Message-ID: > Thoughts or examples? > dateutil.rrule is what you may use e.g. In [38]: from dateutil import rrule In [39]: from datetime import date In [40]: end = date(2017, 12, 31) In [41]: rr = rrule.rrule(rrule.WEEKLY, byweekday=[0, 2], until=end) In [42]: days = list(rr) In [43]: len(days) Out[43]: 53 In [44]: days[:5], days[-5:] Out[44]: ([datetime.datetime(2017, 6, 28, 23, 58, 11), datetime.datetime(2017, 7, 3, 23, 58, 11), datetime.datetime(2017, 7, 5, 23, 58, 11), datetime.datetime(2017, 7, 10, 23, 58, 11), datetime.datetime(2017, 7, 12, 23, 58, 11)], [datetime.datetime(2017, 12, 13, 23, 58, 11), datetime.datetime(2017, 12, 18, 23, 58, 11), datetime.datetime(2017, 12, 20, 23, 58, 11), datetime.datetime(2017, 12, 25, 23, 58, 11), datetime.datetime(2017, 12, 27, 23, 58, 11)]) In [45]: From dhariyalbhaskar at gmail.com Wed Jun 28 02:11:53 2017 From: dhariyalbhaskar at gmail.com (Bhaskar Dhariyal) Date: Tue, 27 Jun 2017 23:11:53 -0700 (PDT) Subject: Combining 2 data series into one Message-ID: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> Hi! I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I want to make it as df['name']. How to do it using pandas dataframe. first_name ---------- bhaskar Rohit last_name ----------- dhariyal Gavval should appear as name ---------- bhaskar dhariyal Rohit Gavval Thanks From renting at astron.nl Wed Jun 28 03:51:17 2017 From: renting at astron.nl (Adriaan Renting) Date: Wed, 28 Jun 2017 09:51:17 +0200 Subject: Syntax error for simple script In-Reply-To: <5134b7c7-f97b-d1a0-2743-4af18964e8c2@lucidity.plus.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> <5134b7c7-f97b-d1a0-2743-4af18964e8c2@lucidity.plus.com> Message-ID: <59537C150200001B00018700@smtp1.astron.nl> Adriaan Renting | Email: renting at astron.nl Software Engineer Radio Observatory ASTRON | Phone: +31 521 595 100 (797 direct) P.O. Box 2 | GSM: +31 6 24 25 17 28 NL-7990 AA Dwingeloo | FAX: +31 521 595 101 The Netherlands | Web: http://www.astron.nl/~renting/ >>> On 27-6-2017 at 21:38, Erik wrote: [snip] > > One other possibility is that a beginner fires up Py3 and gets that > "weird" (to them) error when they tried their tutorial or SO code and > then finds out that if they fire up Py2 instead it just works. Now they > might continue to use Py2 because it's "the one that works" for them. > > I would suggest that making the error dissuade them from doing that (for > the cost of a few tens of bytes) is a good thing. > >>> I think the suggestion above is a bit wordy, but even if it was >>> something like "Missing parentheses in call to the 'print' function" >> >> I think this is a good example of "the curse of knowledge". Its hard for > expert > to think like a non-expert. > > [snip] > >> But to the beginner, adding "function" at the error might as well be >> >> Missing parentheses in call to the 'print' wharrgarbl. > > You are concentrating on the detail of what I wrote. I don't care what > the eventual wording is (and for sure, someone else is far more > qualified than me to suggest wording that is better for a beginner). My > point is simply that the error _could_ contain _some_ more information > that addresses this issue for a beginner who may be typing in Py2 > examples that encourages them to seek out Py3 tutorials instead. > >> That's why I asked Ben if there was something we >> could do to make the sentence clearer. > > Exactly. Ben is an example of someone more qualified than me to suggest > the correct words. > >>> Either that or just make it "SyntaxError: invalid syntax" like all the >>> others. >> >> We've been there: adding the specific print message is new to Python 3.5 (or >> maybe 3.4, I forget which). Python 3.3 gives just "SyntaxError: invalid >> syntax". > > That surprises me, to be honest. I had presumed that the parser had just > had the machinery for that message placed where the 'print' keyword used > to be handled rather than ripping the whole thing out when it became a > function. Interesting that it's a retrospective special case. > > E. I agree in general with Erik. I don't like the general SyntaxError. I like more specific errors that you can at least google for. I don't think you need the whole history in the error, but it would be nice if it was specific enough that if you put it in a search engine you get relevant results explaining things. A. From steve at pearwood.info Wed Jun 28 04:55:14 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 28 Jun 2017 08:55:14 GMT Subject: Syntax error for simple script References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> <5134b7c7-f97b-d1a0-2743-4af18964e8c2@lucidity.plus.com> <59537C150200001B00018700@smtp1.astron.nl> Message-ID: <59536ef2$0$21718$c3e8da3@news.astraweb.com> On Wed, 28 Jun 2017 09:51:17 +0200, Adriaan Renting wrote: > it would be nice if it was specific enough that if you put it in a > search engine you get relevant results explaining things. You mean like this? https://duckduckgo.com/?q=SyntaxError%3A+Missing+parentheses+in+call+to+%27print%27 https://www.google.com.au/search?q=SyntaxError%3A+Missing+parentheses+in+call+to+%27print%27 :-) -- Steve From greg.ewing at canterbury.ac.nz Wed Jun 28 05:11:25 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 28 Jun 2017 21:11:25 +1200 Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: <5953144a$0$22140$c3e8da3$5496439d@news.astraweb.com> References: <87r2y5xm11.fsf@elektro.pacujo.net> <87injhxjsj.fsf@elektro.pacujo.net> <5953144a$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > It should be obvious that if you allow the use of external > libraries that can contain arbitrary amounts of code, *without* counting that > external code towards your measure of code complexity, you get a bogus > measurement of code complexity. Numpy isn't really doing a lot here, just saving you from having to write some explicit loops. The code would be maybe 2 or 3 times bigger at most without it. What this example says to me is not so much how powerful Python is, but how simple in essence neural networks are. -- Greg From paul.james.barry at gmail.com Wed Jun 28 05:13:06 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Wed, 28 Jun 2017 10:13:06 +0100 Subject: Combining 2 data series into one In-Reply-To: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> References: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> Message-ID: This should do it: >>> import pandas as pd >>> >>> df1 = pd.DataFrame(['bhaskar', 'Rohit'], columns=['first_name']) >>> df1 first_name 0 bhaskar 1 Rohit >>> df2 = pd.DataFrame(['dhariyal', 'Gavval'], columns=['last_name']) >>> df2 last_name 0 dhariyal 1 Gavval >>> df = pd.DataFrame() >>> df['name'] = df1['first_name'] + ' ' + df2['last_name'] >>> df name 0 bhaskar dhariyal 1 Rohit Gavval >>> Again, I draw your attention to Jake VanderPlas's excellent book, which is available for free on the web. All of these kind of data manipulations are covered there: https://github.com/jakevdp/PythonDataScienceHandbook - the hard copy is worth owning too (if you plan to do a lot of work using numpy/pandas). I'd also recommend the upcoming 2nd edition of Wes McKinney's "Python for Data Analysis" book - I've just finished tech reviewing it for O'Reilly, and it is very good, too - highly recommended. Regards. Paul. On 28 June 2017 at 07:11, Bhaskar Dhariyal wrote: > Hi! > > I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I want to > make it as df['name']. How to do it using pandas dataframe. > > first_name > ---------- > bhaskar > Rohit > > > last_name > ----------- > dhariyal > Gavval > > should appear as > > name > ---------- > bhaskar dhariyal > Rohit Gavval > > > > Thanks > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From dhariyalbhaskar at gmail.com Wed Jun 28 07:19:20 2017 From: dhariyalbhaskar at gmail.com (Bhaskar Dhariyal) Date: Wed, 28 Jun 2017 04:19:20 -0700 (PDT) Subject: Combining 2 data series into one In-Reply-To: References: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> Message-ID: <3ea981f5-74fb-4f4c-b126-4b5f31c245ba@googlegroups.com> On Wednesday, 28 June 2017 14:43:48 UTC+5:30, Paul Barry wrote: > This should do it: > > >>> import pandas as pd > >>> > >>> df1 = pd.DataFrame(['bhaskar', 'Rohit'], columns=['first_name']) > >>> df1 > first_name > 0 bhaskar > 1 Rohit > >>> df2 = pd.DataFrame(['dhariyal', 'Gavval'], columns=['last_name']) > >>> df2 > last_name > 0 dhariyal > 1 Gavval > >>> df = pd.DataFrame() > >>> df['name'] = df1['first_name'] + ' ' + df2['last_name'] > >>> df > name > 0 bhaskar dhariyal > 1 Rohit Gavval > >>> > > Again, I draw your attention to Jake VanderPlas's excellent book, which is > available for free on the web. All of these kind of data manipulations are > covered there: https://github.com/jakevdp/PythonDataScienceHandbook - the > hard copy is worth owning too (if you plan to do a lot of work using > numpy/pandas). > > I'd also recommend the upcoming 2nd edition of Wes McKinney's "Python for > Data Analysis" book - I've just finished tech reviewing it for O'Reilly, > and it is very good, too - highly recommended. > > Regards. > > Paul. > > On 28 June 2017 at 07:11, Bhaskar Dhariyal > wrote: > > > Hi! > > > > I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I want to > > make it as df['name']. How to do it using pandas dataframe. > > > > first_name > > ---------- > > bhaskar > > Rohit > > > > > > last_name > > ----------- > > dhariyal > > Gavval > > > > should appear as > > > > name > > ---------- > > bhaskar dhariyal > > Rohit Gavval > > > > > > > > Thanks > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > > -- > Paul Barry, t: @barrypj - w: > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. https://drive.google.com/open?id=0Bw2Avni0DUa3aFJKdC1Xd2trM2c link to code From caulagi at gmail.com Wed Jun 28 07:46:49 2017 From: caulagi at gmail.com (caulagi at gmail.com) Date: Wed, 28 Jun 2017 04:46:49 -0700 (PDT) Subject: asyncio: no current event loop in thread 'MainThread' Message-ID: <62ea107d-87f2-4b72-bb67-81bc0a253af6@googlegroups.com> I am trying to write a test for a function using asyncio. Can I mix using unittest.TestCase and asyncio.test_utils.TestCase? When I do that, the loop in test_utils.TestCase seems to affect the other code I have. This is a simplified example - $ pip freeze py==1.4.34 pytest==3.1.2 $ cat test_foo.py import asyncio import unittest from asyncio import test_utils async def foo(): return True class Foo1Test(test_utils.TestCase): def setUp(self): super().setUp() self.loop = self.new_test_loop() self.set_event_loop(self.loop) def test_foo(self): res = self.loop.run_until_complete(foo()) assert res is True class Foo2Test(unittest.TestCase): def test_foo(self): loop = asyncio.get_event_loop() res = loop.run_until_complete(foo()) assert res is True class Foo3Test(test_utils.TestCase): def setUp(self): super().setUp() self.loop = self.new_test_loop() self.set_event_loop(self.loop) def test_foo(self): res = self.loop.run_until_complete(foo()) assert res is True $ py.test -v ============================================================================== test session starts =============================================================================== platform darwin -- Python 3.6.1, pytest-3.1.2, py-1.4.34, pluggy-0.4.0 -- /private/tmp/foobar/.env/bin/python3 cachedir: .cache rootdir: /private/tmp/foobar, inifile: collected 3 items test_foo.py::Foo1Test::test_foo PASSED test_foo.py::Foo2Test::test_foo FAILED test_foo.py::Foo3Test::test_foo PASSED ==================================================================================== FAILURES ==================================================================================== _______________________________________________________________________________ Foo2Test.test_foo ________________________________________________________________________________ self = def test_foo(self): > loop = asyncio.get_event_loop() test_foo.py:26: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/events.py:678: in get_event_loop return get_event_loop_policy().get_event_loop() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_event_loop(self): """Get the event loop. This may be None or an instance of EventLoop. """ if (self._local._loop is None and not self._local._set_called and isinstance(threading.current_thread(), threading._MainThread)): self.set_event_loop(self.new_event_loop()) if self._local._loop is None: raise RuntimeError('There is no current event loop in thread %r.' > % threading.current_thread().name) E RuntimeError: There is no current event loop in thread 'MainThread'. /usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/events.py:584: RuntimeError What am I missing? From paul.james.barry at gmail.com Wed Jun 28 07:47:47 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Wed, 28 Jun 2017 12:47:47 +0100 Subject: Combining 2 data series into one In-Reply-To: <3ea981f5-74fb-4f4c-b126-4b5f31c245ba@googlegroups.com> References: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> <3ea981f5-74fb-4f4c-b126-4b5f31c245ba@googlegroups.com> Message-ID: On the line that's failing, your code is this: combinedX=combinedX+dframe['tf'] which uses combinedX on both sides of the assignment statement - note that Python is reporting a 'MemoryError", which may be happening due to this "double use" (maybe). What happens if you create a new dataframe, like this: newX = combinedX + dframe['tf'] Regardless, it looks like you are doing a dataframe merge. Jake V's book has an excellent section on it here: http://nbviewer.jupyter.org/github/jakevdp/PythonDataScienceHandbook/blob/master/notebooks/03.07-Merge-and-Join.ipynb - this should take about 20 minutes to read, and may be of use to you. Paul. On 28 June 2017 at 12:19, Bhaskar Dhariyal wrote: > On Wednesday, 28 June 2017 14:43:48 UTC+5:30, Paul Barry wrote: > > This should do it: > > > > >>> import pandas as pd > > >>> > > >>> df1 = pd.DataFrame(['bhaskar', 'Rohit'], columns=['first_name']) > > >>> df1 > > first_name > > 0 bhaskar > > 1 Rohit > > >>> df2 = pd.DataFrame(['dhariyal', 'Gavval'], columns=['last_name']) > > >>> df2 > > last_name > > 0 dhariyal > > 1 Gavval > > >>> df = pd.DataFrame() > > >>> df['name'] = df1['first_name'] + ' ' + df2['last_name'] > > >>> df > > name > > 0 bhaskar dhariyal > > 1 Rohit Gavval > > >>> > > > > Again, I draw your attention to Jake VanderPlas's excellent book, which > is > > available for free on the web. All of these kind of data manipulations > are > > covered there: https://github.com/jakevdp/PythonDataScienceHandbook - > the > > hard copy is worth owning too (if you plan to do a lot of work using > > numpy/pandas). > > > > I'd also recommend the upcoming 2nd edition of Wes McKinney's "Python for > > Data Analysis" book - I've just finished tech reviewing it for O'Reilly, > > and it is very good, too - highly recommended. > > > > Regards. > > > > Paul. > > > > On 28 June 2017 at 07:11, Bhaskar Dhariyal > > wrote: > > > > > Hi! > > > > > > I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I want > to > > > make it as df['name']. How to do it using pandas dataframe. > > > > > > first_name > > > ---------- > > > bhaskar > > > Rohit > > > > > > > > > last_name > > > ----------- > > > dhariyal > > > Gavval > > > > > > should appear as > > > > > > name > > > ---------- > > > bhaskar dhariyal > > > Rohit Gavval > > > > > > > > > > > > Thanks > > > -- > > > https://mail.python.org/mailman/listinfo/python-list > > > > > > > > > > > -- > > Paul Barry, t: @barrypj - w: > > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > > https://drive.google.com/open?id=0Bw2Avni0DUa3aFJKdC1Xd2trM2c > link to code > -- > https://mail.python.org/mailman/listinfo/python-list > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From paul.james.barry at gmail.com Wed Jun 28 08:02:36 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Wed, 28 Jun 2017 13:02:36 +0100 Subject: Combining 2 data series into one In-Reply-To: References: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> <3ea981f5-74fb-4f4c-b126-4b5f31c245ba@googlegroups.com> Message-ID: Maybe try your code on a sub-set of your data - perhaps 1000 lines of data? - to see if that works. Anyone else on the list suggest anything to try here? On 28 June 2017 at 12:50, Bhaskar Dhariyal wrote: > No it didn't work. I am getting memory error. Using 32GB RAM system > > On Wed, Jun 28, 2017 at 5:17 PM, Paul Barry > wrote: > >> On the line that's failing, your code is this: >> >> combinedX=combinedX+dframe['tf'] >> >> which uses combinedX on both sides of the assignment statement - note >> that Python is reporting a 'MemoryError", which may be happening due to >> this "double use" (maybe). What happens if you create a new dataframe, >> like this: >> >> newX = combinedX + dframe['tf'] >> >> Regardless, it looks like you are doing a dataframe merge. Jake V's book >> has an excellent section on it here: http://nbviewer.jupyter. >> org/github/jakevdp/PythonDataScienceHandbook/blob/master/ >> notebooks/03.07-Merge-and-Join.ipynb - this should take about 20 minutes >> to read, and may be of use to you. >> >> Paul. >> >> >> >> On 28 June 2017 at 12:19, Bhaskar Dhariyal >> wrote: >> >>> On Wednesday, 28 June 2017 14:43:48 UTC+5:30, Paul Barry wrote: >>> > This should do it: >>> > >>> > >>> import pandas as pd >>> > >>> >>> > >>> df1 = pd.DataFrame(['bhaskar', 'Rohit'], columns=['first_name']) >>> > >>> df1 >>> > first_name >>> > 0 bhaskar >>> > 1 Rohit >>> > >>> df2 = pd.DataFrame(['dhariyal', 'Gavval'], columns=['last_name']) >>> > >>> df2 >>> > last_name >>> > 0 dhariyal >>> > 1 Gavval >>> > >>> df = pd.DataFrame() >>> > >>> df['name'] = df1['first_name'] + ' ' + df2['last_name'] >>> > >>> df >>> > name >>> > 0 bhaskar dhariyal >>> > 1 Rohit Gavval >>> > >>> >>> > >>> > Again, I draw your attention to Jake VanderPlas's excellent book, >>> which is >>> > available for free on the web. All of these kind of data >>> manipulations are >>> > covered there: https://github.com/jakevdp/PythonDataScienceHandbook >>> - the >>> > hard copy is worth owning too (if you plan to do a lot of work using >>> > numpy/pandas). >>> > >>> > I'd also recommend the upcoming 2nd edition of Wes McKinney's "Python >>> for >>> > Data Analysis" book - I've just finished tech reviewing it for >>> O'Reilly, >>> > and it is very good, too - highly recommended. >>> > >>> > Regards. >>> > >>> > Paul. >>> > >>> > On 28 June 2017 at 07:11, Bhaskar Dhariyal >>> > wrote: >>> > >>> > > Hi! >>> > > >>> > > I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I >>> want to >>> > > make it as df['name']. How to do it using pandas dataframe. >>> > > >>> > > first_name >>> > > ---------- >>> > > bhaskar >>> > > Rohit >>> > > >>> > > >>> > > last_name >>> > > ----------- >>> > > dhariyal >>> > > Gavval >>> > > >>> > > should appear as >>> > > >>> > > name >>> > > ---------- >>> > > bhaskar dhariyal >>> > > Rohit Gavval >>> > > >>> > > >>> > > >>> > > Thanks >>> > > -- >>> > > https://mail.python.org/mailman/listinfo/python-list >>> > > >>> > >>> > >>> > >>> > -- >>> > Paul Barry, t: @barrypj - w: >>> > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >>> > Lecturer, Computer Networking: Institute of Technology, Carlow, >>> Ireland. >>> >>> https://drive.google.com/open?id=0Bw2Avni0DUa3aFJKdC1Xd2trM2c >>> link to code >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> >> >> >> >> -- >> Paul Barry, t: @barrypj - w: >> http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >> > > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From paul.james.barry at gmail.com Wed Jun 28 08:30:25 2017 From: paul.james.barry at gmail.com (Paul Barry) Date: Wed, 28 Jun 2017 13:30:25 +0100 Subject: Combining 2 data series into one In-Reply-To: References: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> <3ea981f5-74fb-4f4c-b126-4b5f31c245ba@googlegroups.com> Message-ID: Maybe look at using .concat instead of + See: http://nbviewer.jupyter.org/github/jakevdp/PythonDataScienceHandbook/blob/master/notebooks/03.06-Concat-And-Append.ipynb On 28 June 2017 at 13:02, Paul Barry wrote: > > Maybe try your code on a sub-set of your data - perhaps 1000 lines of > data? - to see if that works. > > Anyone else on the list suggest anything to try here? > > On 28 June 2017 at 12:50, Bhaskar Dhariyal > wrote: > >> No it didn't work. I am getting memory error. Using 32GB RAM system >> >> On Wed, Jun 28, 2017 at 5:17 PM, Paul Barry >> wrote: >> >>> On the line that's failing, your code is this: >>> >>> combinedX=combinedX+dframe['tf'] >>> >>> which uses combinedX on both sides of the assignment statement - note >>> that Python is reporting a 'MemoryError", which may be happening due to >>> this "double use" (maybe). What happens if you create a new dataframe, >>> like this: >>> >>> newX = combinedX + dframe['tf'] >>> >>> Regardless, it looks like you are doing a dataframe merge. Jake V's >>> book has an excellent section on it here: http://nbviewer.jupyter. >>> org/github/jakevdp/PythonDataScienceHandbook/blob/master/not >>> ebooks/03.07-Merge-and-Join.ipynb - this should take about 20 minutes >>> to read, and may be of use to you. >>> >>> Paul. >>> >>> >>> >>> On 28 June 2017 at 12:19, Bhaskar Dhariyal >>> wrote: >>> >>>> On Wednesday, 28 June 2017 14:43:48 UTC+5:30, Paul Barry wrote: >>>> > This should do it: >>>> > >>>> > >>> import pandas as pd >>>> > >>> >>>> > >>> df1 = pd.DataFrame(['bhaskar', 'Rohit'], columns=['first_name']) >>>> > >>> df1 >>>> > first_name >>>> > 0 bhaskar >>>> > 1 Rohit >>>> > >>> df2 = pd.DataFrame(['dhariyal', 'Gavval'], columns=['last_name']) >>>> > >>> df2 >>>> > last_name >>>> > 0 dhariyal >>>> > 1 Gavval >>>> > >>> df = pd.DataFrame() >>>> > >>> df['name'] = df1['first_name'] + ' ' + df2['last_name'] >>>> > >>> df >>>> > name >>>> > 0 bhaskar dhariyal >>>> > 1 Rohit Gavval >>>> > >>> >>>> > >>>> > Again, I draw your attention to Jake VanderPlas's excellent book, >>>> which is >>>> > available for free on the web. All of these kind of data >>>> manipulations are >>>> > covered there: https://github.com/jakevdp/PythonDataScienceHandbook >>>> - the >>>> > hard copy is worth owning too (if you plan to do a lot of work using >>>> > numpy/pandas). >>>> > >>>> > I'd also recommend the upcoming 2nd edition of Wes McKinney's "Python >>>> for >>>> > Data Analysis" book - I've just finished tech reviewing it for >>>> O'Reilly, >>>> > and it is very good, too - highly recommended. >>>> > >>>> > Regards. >>>> > >>>> > Paul. >>>> > >>>> > On 28 June 2017 at 07:11, Bhaskar Dhariyal >>> > >>>> > wrote: >>>> > >>>> > > Hi! >>>> > > >>>> > > I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I >>>> want to >>>> > > make it as df['name']. How to do it using pandas dataframe. >>>> > > >>>> > > first_name >>>> > > ---------- >>>> > > bhaskar >>>> > > Rohit >>>> > > >>>> > > >>>> > > last_name >>>> > > ----------- >>>> > > dhariyal >>>> > > Gavval >>>> > > >>>> > > should appear as >>>> > > >>>> > > name >>>> > > ---------- >>>> > > bhaskar dhariyal >>>> > > Rohit Gavval >>>> > > >>>> > > >>>> > > >>>> > > Thanks >>>> > > -- >>>> > > https://mail.python.org/mailman/listinfo/python-list >>>> > > >>>> > >>>> > >>>> > >>>> > -- >>>> > Paul Barry, t: @barrypj - w: >>>> > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >>>> > Lecturer, Computer Networking: Institute of Technology, Carlow, >>>> Ireland. >>>> >>>> https://drive.google.com/open?id=0Bw2Avni0DUa3aFJKdC1Xd2trM2c >>>> link to code >>>> -- >>>> https://mail.python.org/mailman/listinfo/python-list >>>> >>> >>> >>> >>> -- >>> Paul Barry, t: @barrypj - w: >>> http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >>> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >>> >> >> > > > -- > Paul Barry, t: @barrypj - w: > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From mal at europython.eu Wed Jun 28 09:40:02 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Wed, 28 Jun 2017 15:40:02 +0200 Subject: EuroPython 2017: On-desk Rates and Day Passes Message-ID: We will be switching to the on-desk rates for tickets on Monday next week (July 3rd), so this is your last chance to get tickets at the regular rate, which is about 30% less than the on-desk rate. * EuroPython 2017 Tickets * https://ep2017.europython.eu/en/registration/ On-desk Rates ------------- We will have the following three categories of ticket prices for the on-desk full conference tickets (all 8 days): * Personal full ticket: EUR 490.00 incl. 22% VAT * Business full ticket: EUR 720.00 excl. VAT, EUR 878.40 incl. 22% VAT Please note that we do not sell on-desk student tickets. Students who decide late will have to buy day passes or a personal ticket. Day Passes ---------- As in the past, we will also sell day passes at the conference venue. Day passes for the conference (valid for the day when you pick up the badge): * Student day ticket: EUR 55.00 incl. 22% VAT * Personal day ticket: EUR 148.00 incl. 22% VAT * Business day ticket: EUR 215.00 excl. VAT, EUR 262.30 incl. 22% VAT Please see the registration page for full details of what is included in the ticket price. Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/880056812669743104 Thanks. From dhariyalbhaskar at gmail.com Wed Jun 28 10:27:14 2017 From: dhariyalbhaskar at gmail.com (Bhaskar Dhariyal) Date: Wed, 28 Jun 2017 07:27:14 -0700 (PDT) Subject: Combining 2 data series into one In-Reply-To: References: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> <3ea981f5-74fb-4f4c-b126-4b5f31c245ba@googlegroups.com> Message-ID: <4003bc1d-fcc4-46a4-841d-5c88d4f48e9e@googlegroups.com> On Wednesday, 28 June 2017 18:01:19 UTC+5:30, Paul Barry wrote: > Maybe look at using .concat instead of + > > See: > http://nbviewer.jupyter.org/github/jakevdp/PythonDataScienceHandbook/blob/master/notebooks/03.06-Concat-And-Append.ipynb > > On 28 June 2017 at 13:02, Paul Barry wrote: > > > > > Maybe try your code on a sub-set of your data - perhaps 1000 lines of > > data? - to see if that works. > > > > Anyone else on the list suggest anything to try here? > > > > On 28 June 2017 at 12:50, Bhaskar Dhariyal > > wrote: > > > >> No it didn't work. I am getting memory error. Using 32GB RAM system > >> > >> On Wed, Jun 28, 2017 at 5:17 PM, Paul Barry > >> wrote: > >> > >>> On the line that's failing, your code is this: > >>> > >>> combinedX=combinedX+dframe['tf'] > >>> > >>> which uses combinedX on both sides of the assignment statement - note > >>> that Python is reporting a 'MemoryError", which may be happening due to > >>> this "double use" (maybe). What happens if you create a new dataframe, > >>> like this: > >>> > >>> newX = combinedX + dframe['tf'] > >>> > >>> Regardless, it looks like you are doing a dataframe merge. Jake V's > >>> book has an excellent section on it here: http://nbviewer.jupyter. > >>> org/github/jakevdp/PythonDataScienceHandbook/blob/master/not > >>> ebooks/03.07-Merge-and-Join.ipynb - this should take about 20 minutes > >>> to read, and may be of use to you. > >>> > >>> Paul. > >>> > >>> > >>> > >>> On 28 June 2017 at 12:19, Bhaskar Dhariyal > >>> wrote: > >>> > >>>> On Wednesday, 28 June 2017 14:43:48 UTC+5:30, Paul Barry wrote: > >>>> > This should do it: > >>>> > > >>>> > >>> import pandas as pd > >>>> > >>> > >>>> > >>> df1 = pd.DataFrame(['bhaskar', 'Rohit'], columns=['first_name']) > >>>> > >>> df1 > >>>> > first_name > >>>> > 0 bhaskar > >>>> > 1 Rohit > >>>> > >>> df2 = pd.DataFrame(['dhariyal', 'Gavval'], columns=['last_name']) > >>>> > >>> df2 > >>>> > last_name > >>>> > 0 dhariyal > >>>> > 1 Gavval > >>>> > >>> df = pd.DataFrame() > >>>> > >>> df['name'] = df1['first_name'] + ' ' + df2['last_name'] > >>>> > >>> df > >>>> > name > >>>> > 0 bhaskar dhariyal > >>>> > 1 Rohit Gavval > >>>> > >>> > >>>> > > >>>> > Again, I draw your attention to Jake VanderPlas's excellent book, > >>>> which is > >>>> > available for free on the web. All of these kind of data > >>>> manipulations are > >>>> > covered there: https://github.com/jakevdp/PythonDataScienceHandbook > >>>> - the > >>>> > hard copy is worth owning too (if you plan to do a lot of work using > >>>> > numpy/pandas). > >>>> > > >>>> > I'd also recommend the upcoming 2nd edition of Wes McKinney's "Python > >>>> for > >>>> > Data Analysis" book - I've just finished tech reviewing it for > >>>> O'Reilly, > >>>> > and it is very good, too - highly recommended. > >>>> > > >>>> > Regards. > >>>> > > >>>> > Paul. > >>>> > > >>>> > On 28 June 2017 at 07:11, Bhaskar Dhariyal >>>> > > >>>> > wrote: > >>>> > > >>>> > > Hi! > >>>> > > > >>>> > > I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I > >>>> want to > >>>> > > make it as df['name']. How to do it using pandas dataframe. > >>>> > > > >>>> > > first_name > >>>> > > ---------- > >>>> > > bhaskar > >>>> > > Rohit > >>>> > > > >>>> > > > >>>> > > last_name > >>>> > > ----------- > >>>> > > dhariyal > >>>> > > Gavval > >>>> > > > >>>> > > should appear as > >>>> > > > >>>> > > name > >>>> > > ---------- > >>>> > > bhaskar dhariyal > >>>> > > Rohit Gavval > >>>> > > > >>>> > > > >>>> > > > >>>> > > Thanks > >>>> > > -- > >>>> > > https://mail.python.org/mailman/listinfo/python-list > >>>> > > > >>>> > > >>>> > > >>>> > > >>>> > -- > >>>> > Paul Barry, t: @barrypj - w: > >>>> > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > >>>> > Lecturer, Computer Networking: Institute of Technology, Carlow, > >>>> Ireland. > >>>> > >>>> https://drive.google.com/open?id=0Bw2Avni0DUa3aFJKdC1Xd2trM2c > >>>> link to code > >>>> -- > >>>> https://mail.python.org/mailman/listinfo/python-list > >>>> > >>> > >>> > >>> > >>> -- > >>> Paul Barry, t: @barrypj - w: > >>> http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > >>> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > >>> > >> > >> > > > > > > -- > > Paul Barry, t: @barrypj - w: > > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > > > > > > -- > Paul Barry, t: @barrypj - w: > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. I think there is some problem with tfidf matrix From jorge.conrado at cptec.inpe.br Wed Jun 28 10:32:46 2017 From: jorge.conrado at cptec.inpe.br (jorge.conrado at cptec.inpe.br) Date: Wed, 28 Jun 2017 11:32:46 -0300 Subject: Mouse position values Message-ID: <577c023658c3f5298de38807c4e96df2@cptec.inpe.br> Hi, I have 3D data array and would like to plot arbitrary cross section by cliking in my image. I was an IDL user and in IDL we have a cursor.pro that I used to get the X and Y positions on my image. I would like know how can I get the values of the X an Y position for two points (A and B) in my image and save thiese values. Thanks, Conrado From pkpearson at nowhere.invalid Wed Jun 28 11:56:20 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 28 Jun 2017 15:56:20 GMT Subject: A Good Tutorial on Python Decorators References: Message-ID: On Tue, 27 Jun 2017 16:43:38 +0000, Andre M?ller wrote: > Peter Pearson schrieb am Di., 27. Juni 2017 um > 18:35 Uhr: > >> On Tue, 27 Jun 2017 15:10:53 +0000 (UTC), Saurabh Chaturvedi wrote: >> > https://opensource.google.com/projects/py-decorators-tutorial >> >> "No Results found." >> > Activate JavaScript, then you can see the content. > I had the same problem. (Blushing) Thanks. Life is getting difficult for us JavaScript paranoids. -- To email me, substitute nowhere->runbox, invalid->com. From grant.b.edwards at gmail.com Wed Jun 28 12:39:41 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 28 Jun 2017 16:39:41 +0000 (UTC) Subject: Mouse position values References: <577c023658c3f5298de38807c4e96df2@cptec.inpe.br> Message-ID: On 2017-06-28, jorge.conrado at cptec.inpe.br wrote: > I have 3D data array and would like to plot arbitrary cross section > by cliking in my image. I was an IDL user and in IDL we have a > cursor.pro that I used to get the X and Y positions on my image. I > would like know how can I get the values of the X an Y position for > two points (A and B) in my image and save thiese values. Are we supposed to just guess what OS, windowing system, and GUI toolkit you're using? Or are we supposed to tell you which OS, window system, and GUI toolkit to use? -- Grant Edwards grant.b.edwards Yow! I am a jelly donut. at I am a jelly donut. gmail.com From sjeik_appie at hotmail.com Wed Jun 28 14:13:18 2017 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Wed, 28 Jun 2017 18:13:18 +0000 Subject: Combining 2 data series into one In-Reply-To: References: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> <3ea981f5-74fb-4f4c-b126-4b5f31c245ba@googlegroups.com> , Message-ID: (sorry for top posting) Yes, I'd try pd.concat([df1, df2]). Or this: df['both_names'] = df.apply(lambda row: row.name + ' ' + row.surname, axis=1) ________________________________ From: Python-list on behalf of Paul Barry Sent: Wednesday, June 28, 2017 12:30:25 PM To: Bhaskar Dhariyal Cc: python-list at python.org Subject: Re: Combining 2 data series into one Maybe look at using .concat instead of + See: http://nbviewer.jupyter.org/github/jakevdp/PythonDataScienceHandbook/blob/master/notebooks/03.06-Concat-And-Append.ipynb On 28 June 2017 at 13:02, Paul Barry wrote: > > Maybe try your code on a sub-set of your data - perhaps 1000 lines of > data? - to see if that works. > > Anyone else on the list suggest anything to try here? > > On 28 June 2017 at 12:50, Bhaskar Dhariyal > wrote: > >> No it didn't work. I am getting memory error. Using 32GB RAM system >> >> On Wed, Jun 28, 2017 at 5:17 PM, Paul Barry >> wrote: >> >>> On the line that's failing, your code is this: >>> >>> combinedX=combinedX+dframe['tf'] >>> >>> which uses combinedX on both sides of the assignment statement - note >>> that Python is reporting a 'MemoryError", which may be happening due to >>> this "double use" (maybe). What happens if you create a new dataframe, >>> like this: >>> >>> newX = combinedX + dframe['tf'] >>> >>> Regardless, it looks like you are doing a dataframe merge. Jake V's >>> book has an excellent section on it here: http://nbviewer.jupyter. >>> org/github/jakevdp/PythonDataScienceHandbook/blob/master/not >>> ebooks/03.07-Merge-and-Join.ipynb - this should take about 20 minutes >>> to read, and may be of use to you. >>> >>> Paul. >>> >>> >>> >>> On 28 June 2017 at 12:19, Bhaskar Dhariyal >>> wrote: >>> >>>> On Wednesday, 28 June 2017 14:43:48 UTC+5:30, Paul Barry wrote: >>>> > This should do it: >>>> > >>>> > >>> import pandas as pd >>>> > >>> >>>> > >>> df1 = pd.DataFrame(['bhaskar', 'Rohit'], columns=['first_name']) >>>> > >>> df1 >>>> > first_name >>>> > 0 bhaskar >>>> > 1 Rohit >>>> > >>> df2 = pd.DataFrame(['dhariyal', 'Gavval'], columns=['last_name']) >>>> > >>> df2 >>>> > last_name >>>> > 0 dhariyal >>>> > 1 Gavval >>>> > >>> df = pd.DataFrame() >>>> > >>> df['name'] = df1['first_name'] + ' ' + df2['last_name'] >>>> > >>> df >>>> > name >>>> > 0 bhaskar dhariyal >>>> > 1 Rohit Gavval >>>> > >>> >>>> > >>>> > Again, I draw your attention to Jake VanderPlas's excellent book, >>>> which is >>>> > available for free on the web. All of these kind of data >>>> manipulations are >>>> > covered there: https://github.com/jakevdp/PythonDataScienceHandbook >>>> - the >>>> > hard copy is worth owning too (if you plan to do a lot of work using >>>> > numpy/pandas). >>>> > >>>> > I'd also recommend the upcoming 2nd edition of Wes McKinney's "Python >>>> for >>>> > Data Analysis" book - I've just finished tech reviewing it for >>>> O'Reilly, >>>> > and it is very good, too - highly recommended. >>>> > >>>> > Regards. >>>> > >>>> > Paul. >>>> > >>>> > On 28 June 2017 at 07:11, Bhaskar Dhariyal >>> > >>>> > wrote: >>>> > >>>> > > Hi! >>>> > > >>>> > > I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I >>>> want to >>>> > > make it as df['name']. How to do it using pandas dataframe. >>>> > > >>>> > > first_name >>>> > > ---------- >>>> > > bhaskar >>>> > > Rohit >>>> > > >>>> > > >>>> > > last_name >>>> > > ----------- >>>> > > dhariyal >>>> > > Gavval >>>> > > >>>> > > should appear as >>>> > > >>>> > > name >>>> > > ---------- >>>> > > bhaskar dhariyal >>>> > > Rohit Gavval >>>> > > >>>> > > >>>> > > >>>> > > Thanks >>>> > > -- >>>> > > https://mail.python.org/mailman/listinfo/python-list >>>> > > >>>> > >>>> > >>>> > >>>> > -- >>>> > Paul Barry, t: @barrypj - w: >>>> > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >>>> > Lecturer, Computer Networking: Institute of Technology, Carlow, >>>> Ireland. >>>> >>>> https://drive.google.com/open?id=0Bw2Avni0DUa3aFJKdC1Xd2trM2c >>>> link to code >>>> -- >>>> https://mail.python.org/mailman/listinfo/python-list >>>> >>> >>> >>> >>> -- >>> Paul Barry, t: @barrypj - w: >>> http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie >>> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >>> >> >> > > > -- > Paul Barry, t: @barrypj - w: > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > -- Paul Barry, t: @barrypj - w: http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. -- https://mail.python.org/mailman/listinfo/python-list From djd at debrunners.com Wed Jun 28 15:41:04 2017 From: djd at debrunners.com (djd at debrunners.com) Date: Wed, 28 Jun 2017 12:41:04 -0700 (PDT) Subject: memoryview handling deallocated memory Message-ID: <6bccfd02-06ba-4886-9cb6-64fdf9237e2d@googlegroups.com> >From C I'm creating a memoryview object that wraps memory that is not under my control and will be deallocated when the C function creating the memoryview and passing it into Python returns. In Python 3.5 I can invoke `memoryview.release()` to handle this situation but I'm not seeing any equivalent in Python 2.7 (and I need to support both environments). Is there some alternate approach for Python 2.7? I've thought about accessing the Py_buffer and updating it to have zero length and NULL pointer, but that doesn't feel right. TIA. From saxri89 at gmail.com Wed Jun 28 15:52:02 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Wed, 28 Jun 2017 12:52:02 -0700 (PDT) Subject: DJANGO cannot import name _compare_digest Message-ID: <78efb412-7b8f-4aea-926f-e4f418805c45@googlegroups.com> hello i have python 7.13 DJANGO 1.11.2 version django-crypto 0.20 version on windows 10. i want to create a DJANGO app but anytime to try to migrate i take that error : from django.utils.crypto import get_random_string File "C:\Python27\lib\site-packages\django\utils\crypto.py", line 8, in import hmac File "C:\Python27\Lib\hmac.py", line 8, in from operator import _compare_digest as compare_digest ImportError: cannot import name _compare_digest how to fix it ? From Ken.Lewis at trinityhealth.org Wed Jun 28 16:08:00 2017 From: Ken.Lewis at trinityhealth.org (Ken R. Lewis) Date: Wed, 28 Jun 2017 20:08:00 +0000 Subject: Error Message-ID: Hello! I am running a script, and it comes up with an error. What can I do to make the error be corrected? I am thinking it is something with the data, possibly. I have only had Python for a few days, so maybe I copied some codes wrong. Also, does Python have a way to save files downloaded from web sites? Thanks, Ken The error is on this line: data = response.json() Here is the script: #Need to install requests package for python #easy_install requests import requests # Set the request parameters url = 'https://website /file' # Eg. User name="admin", Password="admin" for this code sample. user = 'admim' pwd = 'admin' # Set proper headers #headers = {"Content-Type":"application/xml","Accept":"*/*"} # Do the HTTP request response = requests.get(url, auth=(user, pwd)) #, headers=headers ) print('Encoding:', response.encoding) print('headers:', response.headers) # Check for HTTP codes other than 200 if response.status_code != 200: print('Status:', response.status_code, 'Headers:', response.headers, 'Error Response:',response.json()) exit() # Decode the JSON response into a dictionary and use the data data = response.json() print(data) Here is the error: Encoding: UTF-8 Traceback (most recent call last): File "I:/CernerProcesses/ServiceNow/new0628.py", line 31, in data = response.json() File "C:\Program Files\Python36\lib\site-packages\requests-2.18.1-py3.6.egg\requests\models.py", line 894, in json return complexjson.loads(self.text, **kwargs) File "C:\Program Files\Python36\lib\json\__init__.py", line 354, in loads return _default_decoder.decode(s) File "C:\Program Files\Python36\lib\json\decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Program Files\Python36\lib\json\decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Ken Lewis Trinity Health I.T. Programmer Lead (701) 858-6423 Cell (701)833-8234 Fax (701) 858-6407 Ken.Lewis at trinityhealth.org ________________________________ Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is privileged, confidential, or otherwise exempt from disclosure under applicable law. Any unauthorized review, copy, use, disclosure or distribution is prohibited. If you have received this communication in error, please delete it from your system without copying or forwarding it and notify the sender of the error by reply e-mail. From tjol at tjol.eu Wed Jun 28 16:31:32 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 28 Jun 2017 22:31:32 +0200 Subject: DJANGO cannot import name _compare_digest In-Reply-To: <78efb412-7b8f-4aea-926f-e4f418805c45@googlegroups.com> References: <78efb412-7b8f-4aea-926f-e4f418805c45@googlegroups.com> Message-ID: <595412e0$0$1715$e4fe514c@news.kpn.nl> On 28/06/17 21:52, Xristos Xristoou wrote: > > > hello i have python 7.13 > > DJANGO 1.11.2 version > > django-crypto 0.20 version on windows 10. > > i want to create a DJANGO app but anytime to try to migrate i take > > that error : > > from django.utils.crypto import get_random_string > File "C:\Python27\lib\site-packages\django\utils\crypto.py", line 8, in > import hmac > File "C:\Python27\Lib\hmac.py", line 8, in > from operator import _compare_digest as compare_digest > ImportError: cannot import name _compare_digest > > how to fix it ? > Do you have a file called "operator.py"? If so, it's shadowing the standard library module "operator". Rename it. From saxri89 at gmail.com Wed Jun 28 16:40:53 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Wed, 28 Jun 2017 13:40:53 -0700 (PDT) Subject: DJANGO cannot import name _compare_digest In-Reply-To: <78efb412-7b8f-4aea-926f-e4f418805c45@googlegroups.com> References: <78efb412-7b8f-4aea-926f-e4f418805c45@googlegroups.com> Message-ID: <742121c3-4a51-4091-802a-8e7d65a0979d@googlegroups.com> i dont have 'operator.py' in my system only 'test_operator.py' and 'fix_operator.py' if : import sys print sys.modules['operator'] i get this : that say me how to rename it? From nathan.ernst at gmail.com Wed Jun 28 16:51:11 2017 From: nathan.ernst at gmail.com (Nathan Ernst) Date: Wed, 28 Jun 2017 15:51:11 -0500 Subject: Error In-Reply-To: References: Message-ID: Not sure if this is the cause of your error, but the value for the variable "user" is misspelled according to the preceding comment. "admim" vs "admin" (not the M instead of an N at the end). Regards, Nathan On Wed, Jun 28, 2017 at 3:08 PM, Ken R. Lewis wrote: > Hello! > > I am running a script, and it comes up with an error. What can I do to > make the error be corrected? I am thinking it is something with the data, > possibly. I have only had Python for a few days, so maybe I copied some > codes wrong. Also, does Python have a way to save files downloaded from > web sites? > > Thanks, > Ken > > > The error is on this line: > data = response.json() > > Here is the script: > > #Need to install requests package for python > #easy_install requests > import requests > # Set the request parameters > url = 'https://website /file' > # Eg. User name="admin", Password="admin" for this code sample. > user = 'admim' > pwd = 'admin' > > # Set proper headers > #headers = {"Content-Type":"application/xml","Accept":"*/*"} > # Do the HTTP request > response = requests.get(url, auth=(user, pwd)) #, headers=headers ) > > print('Encoding:', response.encoding) > > print('headers:', response.headers) > > # Check for HTTP codes other than 200 > if response.status_code != 200: > print('Status:', response.status_code, 'Headers:', response.headers, > 'Error Response:',response.json()) > exit() > > # Decode the JSON response into a dictionary and use the data > data = response.json() > print(data) > > Here is the error: > Encoding: UTF-8 > > > Traceback (most recent call last): > File "I:/CernerProcesses/ServiceNow/new0628.py", line 31, in > data = response.json() > File "C:\Program Files\Python36\lib\site-packages\requests-2.18.1-py3.6.egg\requests\models.py", > line 894, in json > return complexjson.loads(self.text, **kwargs) > File "C:\Program Files\Python36\lib\json\__init__.py", line 354, in > loads > return _default_decoder.decode(s) > File "C:\Program Files\Python36\lib\json\decoder.py", line 339, in > decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "C:\Program Files\Python36\lib\json\decoder.py", line 357, in > raw_decode > raise JSONDecodeError("Expecting value", s, err.value) from None > json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) > > > > > Ken Lewis > Trinity Health I.T. > Programmer Lead > (701) 858-6423 > Cell (701)833-8234 > Fax (701) 858-6407 > Ken.Lewis at trinityhealth.org > > > ________________________________ > > Confidentiality Notice: This e-mail message, including any attachments, is > for the sole use of the intended recipient(s) and may contain information > that is privileged, confidential, or otherwise exempt from disclosure under > applicable law. Any unauthorized review, copy, use, disclosure or > distribution is prohibited. If you have received this communication in > error, please delete it from your system without copying or forwarding it > and notify the sender of the error by reply e-mail. > -- > https://mail.python.org/mailman/listinfo/python-list > From steve+python at pearwood.info Wed Jun 28 17:59:37 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 29 Jun 2017 07:59:37 +1000 Subject: A Good Tutorial on Python Decorators References: Message-ID: <595426cb$0$1604$c3e8da3$5496439d@news.astraweb.com> On Thu, 29 Jun 2017 01:56 am, Peter Pearson wrote: > (Blushing) Thanks. Life is getting difficult for us JavaScript paranoids. Its not paranoia if they're really out to get you. https://www.cnet.com/news/javascript-opens-doors-to-browser-based-attacks/ https://www.proofpoint.com/us/corporate-blog/post/adgholas-malvertising-campaign http://www.pcworld.com/article/2604480/malicious-advertising-hits-amazon-youtube-and-yahoo-cisco-says.html When you run Javascript in your browser, you allow anonymous, untrusted third-parties to run code on your PC. And the Javascript sandbox is frankly not enough to prevent them from being malicious, whether they are tracking and spying on you, to outright drive-by exploits that install ransomware and other malware. The Internet is fundamentally broken and hostile, and we're addicted to it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From python at lucidity.plus.com Wed Jun 28 18:00:30 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 28 Jun 2017 23:00:30 +0100 Subject: Syntax error for simple script In-Reply-To: <595318a3$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <7ce8cd8b-dc6f-4ac1-8172-84e9f70a1799@googlegroups.com> <59512b5f$0$1607$c3e8da3$5496439d@news.astraweb.com> <81289114-6f14-4a63-b487-d509842932ac@googlegroups.com> <59526630$0$1619$c3e8da3$5496439d@news.astraweb.com> <5134b7c7-f97b-d1a0-2743-4af18964e8c2@lucidity.plus.com> <595318a3$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <30a6a775-a51d-86f4-8322-a227f8c83d74@lucidity.plus.com> On 28/06/17 03:46, Steve D'Aprano wrote: > All of this is irrelevant if beginners don't read the error message. "if". > End-users > don't read error messages, no matter how well written they are, Reference? > and every word > you add probably cuts the number of people reading it by half. "probably". > That's not something needed in the error message > itself (imagine how annoying it would be if ZeroDivisionError tried to explain > the basis of arithmetic and why you can't divide by zero), Nobody is suggesting that. You are blowing things out of proportion. We are talking about a much more likely scenario that has been brought about by the very specific transition of "print" from a statement to a function between Python versions while there are still many tutorials and how-to guides online which will cite the earlier, obsolete syntax. > and rather than > helping the newbie, it will probably intimidate "probably" > When designing an error message, imagine that you are going to read it a dozen > times a day, every day, for the rest of your career. I did. I thought the suggestion (current text, followed by a bit more information for those who don't immediately recognise the message) was a reasonable compromise. Are you honestly suggesting that people with a career in programming would make this same mistake a dozen times a day, every day? As I said in my previous email, I would be _very_ surprised if this particular error is generated at all outside of the "beginner typing in a Py2 example" and "person who flips between the two versions and forgets occasionally to add the parentheses in Py3". I presumed we were being a bit more pragmatic about this specific issue rather than talking about general theory of error messages. There is presumably a reason why this error was specifically introduced for Py3.[45] as you have previously asserted. >> I would suggest that making the error dissuade them from doing that (for >> the cost of a few tens of bytes) is a good thing. > > The cost isn't ten bytes. The cost is human attention span. I do not understand what you are alluding to here. > Don't think of the newbie reading the error for the first time, because that > only happens *once*. Think of them reading it for the 50th time. Do you think > they will still appreciate your little history lesson? My "little history lesson"? Wow. > Of course it could. It could also include a link to the Python documentation, a > history of how and why Python 3000 came to be, and a note that if you email the > Python Software Foundation they'll pay you $50, and none of these things will > make the slightest difference if people don't read them. Steven, you do this a lot. You take someone's comments and then blow it out of proportion and say ridiculous things in response "Hey, let's just include the kitchen sink, shall we?". Do you realise that those people are also trying to be productive and helpful? Putting people down like this is not respectful of the thought and effort they have put in to make the comment in the first place. I'm not saying you shouldn't point out if you think someone is wrong, but there are ways of doing it. Regards, E. From darcy at vex.net Wed Jun 28 18:20:46 2017 From: darcy at vex.net (D'Arcy Cain) Date: Wed, 28 Jun 2017 18:20:46 -0400 Subject: A Good Tutorial on Python Decorators In-Reply-To: <595426cb$0$1604$c3e8da3$5496439d@news.astraweb.com> References: <595426cb$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 06/28/17 17:59, Steve D'Aprano wrote: > On Thu, 29 Jun 2017 01:56 am, Peter Pearson wrote: > >> (Blushing) Thanks. Life is getting difficult for us JavaScript paranoids. > > Its not paranoia if they're really out to get you. > > https://www.cnet.com/news/javascript-opens-doors-to-browser-based-attacks/ > > https://www.proofpoint.com/us/corporate-blog/post/adgholas-malvertising-campaign > > http://www.pcworld.com/article/2604480/malicious-advertising-hits-amazon-youtube-and-yahoo-cisco-says.html Anyone else find it sad that all of these sites us Javascript? -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From sohcahtoa82 at gmail.com Wed Jun 28 18:27:22 2017 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Wed, 28 Jun 2017 15:27:22 -0700 (PDT) Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: References: Message-ID: <0efcc78a-b29f-4bda-be28-b6b93bf7be97@googlegroups.com> On Tuesday, June 27, 2017 at 12:31:49 PM UTC-7, John Ladasky wrote: > On Tuesday, June 27, 2017 at 9:24:07 AM UTC-7, Sam Chats wrote: > > https://medium.com/technology-invention-and-more/how-to-build-a-simple-neural-network-in-9-lines-of-python-code-cc8f23647ca1 > > OK, that's cheating a bit, using Numpy. It's a nice little program, but it leverages a huge, powerful library. Agreed. It's equivalent to saying "Hey you can create an HTTP server in only 3 lines!" when you're just making a call to SimpleHTTPServer. If I create a AAA-quality game and pack it up so all I have to do is "import mygame; mygame.start()", then have I created a AAA-quality game in only two lines? From walters.justin01 at gmail.com Wed Jun 28 18:39:50 2017 From: walters.justin01 at gmail.com (justin walters) Date: Wed, 28 Jun 2017 15:39:50 -0700 Subject: Error In-Reply-To: References: Message-ID: On Jun 28, 2017 1:47 PM, "Ken R. Lewis" wrote: Hello! I am running a script, and it comes up with an error. What can I do to make the error be corrected? I am thinking it is something with the data, possibly. I have only had Python for a few days, so maybe I copied some codes wrong. Also, does Python have a way to save files downloaded from web sites? Thanks, Ken The error is on this line: data = response.json() Here is the script: #Need to install requests package for python #easy_install requests import requests # Set the request parameters url = 'https://website /file' # Eg. User name="admin", Password="admin" for this code sample. user = 'admim' pwd = 'admin' # Set proper headers #headers = {"Content-Type":"application/xml","Accept":"*/*"} # Do the HTTP request response = requests.get(url, auth=(user, pwd)) #, headers=headers ) print('Encoding:', response.encoding) print('headers:', response.headers) # Check for HTTP codes other than 200 if response.status_code != 200: print('Status:', response.status_code, 'Headers:', response.headers, 'Error Response:',response.json()) exit() # Decode the JSON response into a dictionary and use the data data = response.json() print(data) Here is the error: Encoding: UTF-8 Traceback (most recent call last): File "I:/CernerProcesses/ServiceNow/new0628.py", line 31, in data = response.json() File "C:\Program Files\Python36\lib\site-packages\requests-2.18.1-py3.6.egg\requests\models.py", line 894, in json return complexjson.loads(self.text, **kwargs) File "C:\Program Files\Python36\lib\json\__init__.py", line 354, in loads return _default_decoder.decode(s) File "C:\Program Files\Python36\lib\json\decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Program Files\Python36\lib\json\decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Ken Lewis Trinity Health I.T. Programmer Lead (701) 858-6423 Cell (701)833-8234 Fax (701) 858-6407 Ken.Lewis at trinityhealth.org ________________________________ Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is privileged, confidential, or otherwise exempt from disclosure under applicable law. Any unauthorized review, copy, use, disclosure or distribution is prohibited. If you have received this communication in error, please delete it from your system without copying or forwarding it and notify the sender of the error by reply e-mail. -- https://mail.python.org/mailman/listinfo/python-list Looks to me like you're not receiving any data from your request. In effect, the response body is empty. From flebber.crue at gmail.com Wed Jun 28 19:30:42 2017 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Wed, 28 Jun 2017 16:30:42 -0700 (PDT) Subject: Create a list of dates for same day of week in a year In-Reply-To: References: <08dc0aab-d7ea-41e5-bc05-60e3c05c62f0@googlegroups.com> Message-ID: <0287ee94-2f35-47fe-ba53-e3e8ae8e25dc@googlegroups.com> > > Thoughts or examples? > > > dateutil.rrule is what you may use e.g. > > > In [38]: from dateutil import rrule > > In [39]: from datetime import date > > In [40]: end = date(2017, 12, 31) > > In [41]: rr = rrule.rrule(rrule.WEEKLY, byweekday=[0, 2], until=end) > > In [42]: days = list(rr) > > In [43]: len(days) > Out[43]: 53 > > In [44]: days[:5], days[-5:] > Out[44]: > ([datetime.datetime(2017, 6, 28, 23, 58, 11), > datetime.datetime(2017, 7, 3, 23, 58, 11), > datetime.datetime(2017, 7, 5, 23, 58, 11), > datetime.datetime(2017, 7, 10, 23, 58, 11), > datetime.datetime(2017, 7, 12, 23, 58, 11)], > [datetime.datetime(2017, 12, 13, 23, 58, 11), > datetime.datetime(2017, 12, 18, 23, 58, 11), > datetime.datetime(2017, 12, 20, 23, 58, 11), > datetime.datetime(2017, 12, 25, 23, 58, 11), > datetime.datetime(2017, 12, 27, 23, 58, 11)]) > > In [45]: Thanks. I am just researching now the format that has come out. unclear what 58 represents. Cheers Sayth From marko at pacujo.net Wed Jun 28 19:40:51 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 29 Jun 2017 02:40:51 +0300 Subject: A Good Tutorial on Python Decorators References: <595426cb$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: <8760ffy930.fsf@elektro.pacujo.net> Steve D'Aprano : > When you run Javascript in your browser, you allow anonymous, > untrusted third-parties to run code on your PC. Yes. I'm running NoScript on my browser, but I believe JavaScript (or equivalent) is the way forward. I *love* my Turing Tarpit. Aus dem Paradies, das [Turing] uns geschaffen, soll uns niemand vertreiben k?nnen. > The Internet is fundamentally broken and hostile, and we're addicted > to it. The Internet has taken us to the next level. Let's hope Good will prevail. Marko From michael.stemper at gmail.com Wed Jun 28 19:56:13 2017 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Wed, 28 Jun 2017 18:56:13 -0500 Subject: Create a list of dates for same day of week in a year In-Reply-To: <0287ee94-2f35-47fe-ba53-e3e8ae8e25dc@googlegroups.com> References: <08dc0aab-d7ea-41e5-bc05-60e3c05c62f0@googlegroups.com> <0287ee94-2f35-47fe-ba53-e3e8ae8e25dc@googlegroups.com> Message-ID: On 2017-06-28 18:30, Sayth Renshaw wrote: > >>> Thoughts or examples? >>> >> dateutil.rrule is what you may use e.g. >> In [44]: days[:5], days[-5:] >> Out[44]: >> ([datetime.datetime(2017, 6, 28, 23, 58, 11), >> datetime.datetime(2017, 7, 3, 23, 58, 11), >> datetime.datetime(2017, 7, 5, 23, 58, 11), >> datetime.datetime(2017, 7, 10, 23, 58, 11), >> datetime.datetime(2017, 7, 12, 23, 58, 11)], >> [datetime.datetime(2017, 12, 13, 23, 58, 11), >> datetime.datetime(2017, 12, 18, 23, 58, 11), >> datetime.datetime(2017, 12, 20, 23, 58, 11), >> datetime.datetime(2017, 12, 25, 23, 58, 11), >> datetime.datetime(2017, 12, 27, 23, 58, 11)]) >> >> In [45]: > > Thanks. I am just researching now the format that has come out. unclear what 58 represents. On its own, not much. However, is 23:58:11 easier to understand? -- Michael F. Stemper Nostalgia just ain't what it used to be. From dhariyalbhaskar at gmail.com Thu Jun 29 00:34:56 2017 From: dhariyalbhaskar at gmail.com (Bhaskar Dhariyal) Date: Wed, 28 Jun 2017 21:34:56 -0700 (PDT) Subject: Combining 2 data series into one In-Reply-To: References: <0c051266-a672-44e2-982f-2b69376775b2@googlegroups.com> <3ea981f5-74fb-4f4c-b126-4b5f31c245ba@googlegroups.com> ,> Message-ID: On Wednesday, 28 June 2017 23:43:57 UTC+5:30, Albert-Jan Roskam wrote: > (sorry for top posting) > Yes, I'd try pd.concat([df1, df2]). > Or this: > df['both_names'] = df.apply(lambda row: row.name + ' ' + row.surname, axis=1) > ________________________________ > From: Python-list on behalf of Paul Barry > Sent: Wednesday, June 28, 2017 12:30:25 PM > To: Bhaskar Dhariyal > Cc: python-list at python.org > Subject: Re: Combining 2 data series into one > > Maybe look at using .concat instead of + > > See: > http://nbviewer.jupyter.org/github/jakevdp/PythonDataScienceHandbook/blob/master/notebooks/03.06-Concat-And-Append.ipynb > > On 28 June 2017 at 13:02, Paul Barry wrote: > > > > > Maybe try your code on a sub-set of your data - perhaps 1000 lines of > > data? - to see if that works. > > > > Anyone else on the list suggest anything to try here? > > > > On 28 June 2017 at 12:50, Bhaskar Dhariyal > > wrote: > > > >> No it didn't work. I am getting memory error. Using 32GB RAM system > >> > >> On Wed, Jun 28, 2017 at 5:17 PM, Paul Barry > >> wrote: > >> > >>> On the line that's failing, your code is this: > >>> > >>> combinedX=combinedX+dframe['tf'] > >>> > >>> which uses combinedX on both sides of the assignment statement - note > >>> that Python is reporting a 'MemoryError", which may be happening due to > >>> this "double use" (maybe). What happens if you create a new dataframe, > >>> like this: > >>> > >>> newX = combinedX + dframe['tf'] > >>> > >>> Regardless, it looks like you are doing a dataframe merge. Jake V's > >>> book has an excellent section on it here: http://nbviewer.jupyter. > >>> org/github/jakevdp/PythonDataScienceHandbook/blob/master/not > >>> ebooks/03.07-Merge-and-Join.ipynb - this should take about 20 minutes > >>> to read, and may be of use to you. > >>> > >>> Paul. > >>> > >>> > >>> > >>> On 28 June 2017 at 12:19, Bhaskar Dhariyal > >>> wrote: > >>> > >>>> On Wednesday, 28 June 2017 14:43:48 UTC+5:30, Paul Barry wrote: > >>>> > This should do it: > >>>> > > >>>> > >>> import pandas as pd > >>>> > >>> > >>>> > >>> df1 = pd.DataFrame(['bhaskar', 'Rohit'], columns=['first_name']) > >>>> > >>> df1 > >>>> > first_name > >>>> > 0 bhaskar > >>>> > 1 Rohit > >>>> > >>> df2 = pd.DataFrame(['dhariyal', 'Gavval'], columns=['last_name']) > >>>> > >>> df2 > >>>> > last_name > >>>> > 0 dhariyal > >>>> > 1 Gavval > >>>> > >>> df = pd.DataFrame() > >>>> > >>> df['name'] = df1['first_name'] + ' ' + df2['last_name'] > >>>> > >>> df > >>>> > name > >>>> > 0 bhaskar dhariyal > >>>> > 1 Rohit Gavval > >>>> > >>> > >>>> > > >>>> > Again, I draw your attention to Jake VanderPlas's excellent book, > >>>> which is > >>>> > available for free on the web. All of these kind of data > >>>> manipulations are > >>>> > covered there: https://github.com/jakevdp/PythonDataScienceHandbook > >>>> - the > >>>> > hard copy is worth owning too (if you plan to do a lot of work using > >>>> > numpy/pandas). > >>>> > > >>>> > I'd also recommend the upcoming 2nd edition of Wes McKinney's "Python > >>>> for > >>>> > Data Analysis" book - I've just finished tech reviewing it for > >>>> O'Reilly, > >>>> > and it is very good, too - highly recommended. > >>>> > > >>>> > Regards. > >>>> > > >>>> > Paul. > >>>> > > >>>> > On 28 June 2017 at 07:11, Bhaskar Dhariyal >>>> > > >>>> > wrote: > >>>> > > >>>> > > Hi! > >>>> > > > >>>> > > I have 2 dataframe i.e. df1['first_name'] and df2['last_name']. I > >>>> want to > >>>> > > make it as df['name']. How to do it using pandas dataframe. > >>>> > > > >>>> > > first_name > >>>> > > ---------- > >>>> > > bhaskar > >>>> > > Rohit > >>>> > > > >>>> > > > >>>> > > last_name > >>>> > > ----------- > >>>> > > dhariyal > >>>> > > Gavval > >>>> > > > >>>> > > should appear as > >>>> > > > >>>> > > name > >>>> > > ---------- > >>>> > > bhaskar dhariyal > >>>> > > Rohit Gavval > >>>> > > > >>>> > > > >>>> > > > >>>> > > Thanks > >>>> > > -- > >>>> > > https://mail.python.org/mailman/listinfo/python-list > >>>> > > > >>>> > > >>>> > > >>>> > > >>>> > -- > >>>> > Paul Barry, t: @barrypj - w: > >>>> > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > >>>> > Lecturer, Computer Networking: Institute of Technology, Carlow, > >>>> Ireland. > >>>> > >>>> https://drive.google.com/open?id=0Bw2Avni0DUa3aFJKdC1Xd2trM2c > >>>> link to code > >>>> -- > >>>> https://mail.python.org/mailman/listinfo/python-list > >>>> > >>> > >>> > >>> > >>> -- > >>> Paul Barry, t: @barrypj - w: > >>> http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > >>> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > >>> > >> > >> > > > > > > -- > > Paul Barry, t: @barrypj - w: > > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > > > > > > -- > Paul Barry, t: @barrypj - w: > http://paulbarry.itcarlow.ie - e: paul.barry at itcarlow.ie > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > -- > https://mail.python.org/mailman/listinfo/python-list Hi Albert! Thanks for replying. That issue was resolved. But I m struck with a new problem. I generated tfidf representation for pandas dataframe where each row contains some text. I also had some numerical feature which I wanted to combine with tfidf matrix. But this is giving memory error. From steve at pearwood.info Thu Jun 29 02:50:19 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 29 Jun 2017 06:50:19 GMT Subject: sys.exc_info Message-ID: <5954a32b$0$21718$c3e8da3@news.astraweb.com> sys.exc_info() returns three items: (exception type, exception value, traceback) https://docs.python.org/2/library/sys.html#sys.exc_info https://docs.python.org/3/library/sys.html#sys.exc_info and may be used something like this example: try: something except: exc_type, exc, tb = sys.exc_info() print(traceback.extract_tb(tb)) raise Why does it return the exception type separately from the exception, when the type can be derived by calling `type(exc)`? -- Steve From steve at pearwood.info Thu Jun 29 02:55:50 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 29 Jun 2017 06:55:50 GMT Subject: Mouse position values References: <577c023658c3f5298de38807c4e96df2@cptec.inpe.br> Message-ID: <5954a476$0$21718$c3e8da3@news.astraweb.com> On Wed, 28 Jun 2017 11:32:46 -0300, jorge.conrado wrote: > Hi, > > I have 3D data array and would like to plot arbitrary cross section by > cliking in my image. I was an IDL user and in IDL we have a cursor.pro > that I used to get the X and Y positions on my image. I would like know > how can I get the values of the X an Y position for two points (A and B) > in my image and save thiese values. Unlike some other languages, Python doesn't support getting the mouse coordinates directly. You will have to tell us what version of Python you are using, what operating system you are using, and the version of the OS. If you are using Linux, you probably should tell us what Window Manager you have (Gnome, XFCE, KDE, etc...). Are you using a GUI toolkit, like tkinter, PyQt or similar? If so you should tell us. I'm sorry if this seems complicated, but Python is a language designed to operate on hardware where there might not even be a mouse, so it doesn't support this function without using an external library, and that will depend on your system. -- Steve From john_ladasky at sbcglobal.net Thu Jun 29 05:24:15 2017 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Thu, 29 Jun 2017 02:24:15 -0700 (PDT) Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: <87r2y5xm11.fsf@elektro.pacujo.net> References: <87r2y5xm11.fsf@elektro.pacujo.net> Message-ID: On Tuesday, June 27, 2017 at 12:34:46 PM UTC-7, Marko Rauhamaa wrote: > John Ladasky : > > OK, that's cheating a bit, using Numpy. It's a nice little program, > > but it leverages a huge, powerful library. > > What would *not* be cheating? A language without a library would be > dead. Python's standard library is huge, and useful. Numpy is not part of Python's STANDARD library. I love Numpy, and use it in almost everything I write. But I see that many people won't need it, would find it overwhelming, and would find its way of doing things rather different than standard Python (vectorized mathematical operations, fancy slicing, broadcasting, etc.). So to use Numpy in a post that says, "implement a neural network in Python with just nine lines of code!" seems a little misleading to me. Python is IMHO the best programming language in use today -- and it's the best one for a beginner to start learning. But I wouldn't want to oversell the language to a novice programmer. It's going to take you a long time to understand exactly what those nine lines of code are doing, if you're new to this. From john_ladasky at sbcglobal.net Thu Jun 29 05:34:53 2017 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Thu, 29 Jun 2017 02:34:53 -0700 (PDT) Subject: How to build a simple neural network in 9 lines of Python code In-Reply-To: <5953144a$0$22140$c3e8da3$5496439d@news.astraweb.com> References: <87r2y5xm11.fsf@elektro.pacujo.net> <87injhxjsj.fsf@elektro.pacujo.net> <5953144a$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: <74024627-67bb-48b9-a4ab-0fa58047af16@googlegroups.com> On Tuesday, June 27, 2017 at 7:28:58 PM UTC-7, Steve D'Aprano wrote: > On Wed, 28 Jun 2017 06:22 am, Marko Rauhamaa wrote: > > > You saw the APL example, right? APL's standard runtime/library contains > > most of Numpy functionality because that's what APL has been designed > > for. > > > > Is that cheating? > > > Of course not. That demonstrates beautifully (or perhaps "unreadably tersely") > that the APL language primitives are extremely powerful (too powerful?). Interesting how this discussion has detoured into APL. I think that these comments underscore my point that Numpy's way of doing things isn't quite Python's way of doing things, that it's a little esoteric. I don't know APL, but... some years ago, I remember reading that the concept of a scalar did not exist in APL. The equivalent of a scalar in APL is (if I understand correctly) a one-dimensional array with one element in it. When I first read that, I thought it was a needless complication. After ten years of Numpy and three months of TensorFlow, it's starting to dawn on me why that might actually make programming sense... if you're thinking in matrix algebra all the time, which increasingly, I find myself doing. From zulfiqarshah987 at gmail.com Thu Jun 29 06:02:58 2017 From: zulfiqarshah987 at gmail.com (Test Banks) Date: Thu, 29 Jun 2017 03:02:58 -0700 (PDT) Subject: Operations Management 13th E by Stevenson Solution Manuals Message-ID: <968e728c-bc23-4fec-9a91-17504ead2fe3@googlegroups.com> Greetings Students, We do have Solution Manuals and Test Bank for OPERATIONS MANAGEMENT 13TH E BY STEVENSON at reasonable price. You can get above mentioned resources by sending email to pro.fast(@)hotmail(dot)com. Send your order queries at PRO.FAST(@)HOTMAIL(DOT)COM Below are details given for this book Book Name: OPERATIONS MANAGEMENT Authors: William J Stevenson Edition: 13th E ISBN-10: 1259667472 ISBN-13: 9781259667473 Product Format: MS Word Total Chapters: 19 Items available : Solution Manuals / Test Bank / PPT,s Please mention complete details for your book so we could send you sample accordingly. Best Regards, P.S : Please do not post your reply here. We do not monitor queries here. Simply send us an email directly to PRO.FAST (@) HOTMAIL (DOT) COM From zulfiqarshah987 at gmail.com Thu Jun 29 06:03:24 2017 From: zulfiqarshah987 at gmail.com (Test Banks) Date: Thu, 29 Jun 2017 03:03:24 -0700 (PDT) Subject: Test Bank for Corrections in the 21st Century 8th Edition by Frank Schmalleger Message-ID: <1ca80155-810f-49a5-89f0-f6258a2cf000@googlegroups.com> Greetings Students, We do have Test Bank for CORRECTIONS IN THE 21ST CENTURY 8TH EDITION BY SCHMALLEGER at reasonable price. You can get above mentioned resources by sending email to pro.fast(@)hotmail(dot)com Send your order queries at PRO.FAST(@)HOTMAIL(DOT)COM Below are details given for this book Book Name: CORRECTIONS IN THE 21ST CENTURY Authors: Frank Schmalleger, John Smykla Edition: 8th E ISBN-10: 1259916553 ISBN-13: 9781259916557 Product Format: MS Word Total Chapters: 12 Please mention complete details for your book so we could send you sample accordingly. Best Regards, P.S : Please do not post your reply here. We do not monitor queries here. Simply send us an email directly to PRO.FAST (@) HOTMAIL (DOT) COM From zulfiqarshah987 at gmail.com Thu Jun 29 06:03:48 2017 From: zulfiqarshah987 at gmail.com (Test Banks) Date: Thu, 29 Jun 2017 03:03:48 -0700 (PDT) Subject: Solution Manual Test Bank for Financial and Managerial Accounting for MBAs 5th Edition by Easton Message-ID: <38202c95-45e3-435d-b4f9-1099e71de542@googlegroups.com> Greetings Students, We do have Solution Manuals and Test Bank for FINANCIAL AND MANAGERIAL ACCOUNTING FOR MBAs 5TH EDITION BY EASTON at reasonable price. You can get above mentioned resources by sending email to pro.fast(@)hotmail(dot)com Send your order queries at PRO.FAST(@)HOTMAIL(DOT)COM Below are details given for this book Book Name: FINANCIAL AND MANAGERIAL ACCOUNTING FOR MBAs Authors: Peter D. Easton, Robert F. Halsey, Mary Lea McAnally, Al L. Hartgraves, Wayne J. Morse Edition: 5th E ISBN-10: 1618532324 ISBN-13: 9781618532329 Product Format: MS Word Total Modules: 25 Please mention complete details for your book so we could send you sample accordingly. Best Regards, P.S : Please do not post your reply here. We do not monitor queries here. Simply send us an email directly to PRO.FAST (@) HOTMAIL (DOT) COM From zulfiqarshah987 at gmail.com Thu Jun 29 06:04:10 2017 From: zulfiqarshah987 at gmail.com (Test Banks) Date: Thu, 29 Jun 2017 03:04:10 -0700 (PDT) Subject: Solution Manual Test Bank for Financial Accounting for MBAs 7th Edition by Easton Message-ID: <678dbe84-7359-40af-88b6-da35fa525713@googlegroups.com> Greetings Students, We do have Solution Manuals and Test Bank for FINANCIAL ACCOUNTING FOR MBAs 7th EDITION BY EASTON at reasonable price. You can get above mentioned resources by sending email to pro.fast(@)hotmail(dot)com Send your order queries at PRO.FAST(@)HOTMAIL(DOT)COM Below are details given for this book Book Name: FINANCIAL ACCOUNTING FOR MBAs Authors: Peter D. Easton, John J. Wild, Robert F. Halsey, Mary Lea McAnally Edition: 7th E ISBN-13: 9781618532312 Product Format: MS Word Total Modules: 13 Please mention complete details for your book so we could send you sample accordingly. Best Regards, P.S : Please do not post your reply here. We do not monitor queries here. Simply send us an email directly to PRO.FAST (@) HOTMAIL (DOT) COM From zulfiqarshah987 at gmail.com Thu Jun 29 06:04:32 2017 From: zulfiqarshah987 at gmail.com (Test Banks) Date: Thu, 29 Jun 2017 03:04:32 -0700 (PDT) Subject: Solution Manual Test Bank for Financial Statement Analysis and Valuation 5th Edition by Easton Message-ID: <24584100-e7ad-4759-a38e-796029ae0a7b@googlegroups.com> Greetings Students, We do have Solution Manuals and Test Bank for FINANCIAL STATEMENT ANALYSIS AND VALUATION BY EASTON at reasonable price. You can get above mentioned resources by sending email to pro.fast(@)hotmail(dot)com Send your order queries at PRO.FAST(@)HOTMAIL(DOT)COM Below are details given for this book Book Name: FINANCIAL STATEMENT ANALYSIS AND VALUATION Authors: Peter D. Easton, John J. Wild, Robert F. Halsey, Mary Lea McAnally Edition: 5th E ISBN-10: 1618532332 ISBN-13: 9781618532336 Product Format: MS Word Total Modules: 15 + Appendix A,B,C,D Please mention complete details for your book so we could send you sample accordingly. Best Regards, P.S : Please do not post your reply here. We do not monitor queries here. Simply send us an email directly to PRO.FAST (@) HOTMAIL (DOT) COM From rhodri at kynesim.co.uk Thu Jun 29 06:11:36 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 29 Jun 2017 11:11:36 +0100 Subject: Error In-Reply-To: References: Message-ID: <464e52d7-d292-719a-5d1c-2c9a06a96258@kynesim.co.uk> On 28/06/17 21:08, Ken R. Lewis wrote: > Traceback (most recent call last): > File "I:/CernerProcesses/ServiceNow/new0628.py", line 31, in > data = response.json() > File "C:\Program Files\Python36\lib\site-packages\requests-2.18.1-py3.6.egg\requests\models.py", line 894, in json > return complexjson.loads(self.text, **kwargs) > File "C:\Program Files\Python36\lib\json\__init__.py", line 354, in loads > return _default_decoder.decode(s) > File "C:\Program Files\Python36\lib\json\decoder.py", line 339, in decode > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > File "C:\Program Files\Python36\lib\json\decoder.py", line 357, in raw_decode > raise JSONDecodeError("Expecting value", s, err.value) from None > json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) It looks like your reply doesn't contain any JSON. Have you tried dumping the response out to see exactly what was sent? -- Rhodri James *-* Kynesim Ltd From grant.b.edwards at gmail.com Thu Jun 29 10:03:42 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 29 Jun 2017 14:03:42 +0000 (UTC) Subject: Development testing without reinstalling egg constantly? Message-ID: I've forked a copy of https://github.com/Roguelazer/muttdown and have been adding a few features and fixing a few bugs. It's meant to be installed using setup tools, and then invoked via /usr/bin/muttdown which looks like this: #!/usr/lib/python-exec/python2.7/python2 # EASY-INSTALL-ENTRY-SCRIPT: 'muttdown==0.3','console_scripts','muttdown' __requires__ = 'muttdown==0.3' import re import sys from pkg_resources import load_entry_point if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) sys.exit( load_entry_point('muttdown==0.3', 'console_scripts', 'muttdown')() ) The projects 'main.py' can't be run directly from the command line, since it contains code like this: from . import config from . import __version__ __name__ = 'muttdown' [ stuff that does real work ] if __name__ == '__main__': main() I've hacked up the main.py bits shown above to allow it to be run directly in order to test changes without installing, but then I always have to remember to change it back before committing a change. This seems like the wrong way to do things, but I can't figure out what the _right_ way is. What's the Pythonic way to do deal with this? -- Grant Edwards grant.b.edwards Yow! How's the wife? at Is she at home enjoying gmail.com capitalism? From ned at nedbatchelder.com Thu Jun 29 10:52:34 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 29 Jun 2017 07:52:34 -0700 (PDT) Subject: Development testing without reinstalling egg constantly? In-Reply-To: References: Message-ID: <233dc839-0567-439f-8e24-d8659d82ae90@googlegroups.com> On Thursday, June 29, 2017 at 10:04:30 AM UTC-4, Grant Edwards wrote: > I've forked a copy of https://github.com/Roguelazer/muttdown and have > been adding a few features and fixing a few bugs. It's meant to be > installed using setup tools, and then invoked via /usr/bin/muttdown > which looks like this: > > #!/usr/lib/python-exec/python2.7/python2 > # EASY-INSTALL-ENTRY-SCRIPT: 'muttdown==0.3','console_scripts','muttdown' > __requires__ = 'muttdown==0.3' > import re > import sys > from pkg_resources import load_entry_point > > if __name__ == '__main__': > sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) > sys.exit( > load_entry_point('muttdown==0.3', 'console_scripts', 'muttdown')() > ) > > The projects 'main.py' can't be run directly from the command line, > since it contains code like this: > > from . import config > from . import __version__ > __name__ = 'muttdown' > > [ stuff that does real work ] > > if __name__ == '__main__': > main() > > I've hacked up the main.py bits shown above to allow it to be run > directly in order to test changes without installing, but then I > always have to remember to change it back before committing a change. > > This seems like the wrong way to do things, but I can't figure out > what the _right_ way is. What's the Pythonic way to do deal with > this? $ pip install -e . This will install the code in the current directory in "editable" fashion. The files in the current directory *are* the installed code, so when you edit them, you don't have to re-install. --Ned. From __peter__ at web.de Thu Jun 29 11:17:45 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 29 Jun 2017 17:17:45 +0200 Subject: Development testing without reinstalling egg constantly? References: Message-ID: Grant Edwards wrote: > The projects 'main.py' can't be run directly from the command line, > since it contains code like this: > > from . import config > from . import __version__ > __name__ = 'muttdown' > > [ stuff that does real work ] Stupid question: isn't the following > > if __name__ == '__main__': > main() dead code then? Actually, two stupid questions: what is > __name__ = 'muttdown' supposed to achieve in the first place? From grant.b.edwards at gmail.com Thu Jun 29 11:32:21 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 29 Jun 2017 15:32:21 +0000 (UTC) Subject: Development testing without reinstalling egg constantly? References: Message-ID: On 2017-06-29, Peter Otten <__peter__ at web.de> wrote: > Grant Edwards wrote: > >> The projects 'main.py' can't be run directly from the command line, >> since it contains code like this: >> >> from . import config >> from . import __version__ >> __name__ = 'muttdown' >> >> [ stuff that does real work ] > > Stupid question: isn't the following > >> >> if __name__ == '__main__': >> main() > > dead code then? At appears so. > Actually, two stupid questions: what is > >> __name__ = 'muttdown' > > supposed to achieve in the first place? I don't know. This is somebody else's project I'm hacking on, and my understanding of setup tools is pretty much at the "cargo cult" level. -- Grant Edwards grant.b.edwards Yow! FUN is never having to at say you're SUSHI!! gmail.com From info at tundraware.com Thu Jun 29 11:38:46 2017 From: info at tundraware.com (Tim Daneliuk) Date: Thu, 29 Jun 2017 10:38:46 -0500 Subject: Development testing without reinstalling egg constantly? In-Reply-To: References: Message-ID: <1d79f43b-49ad-d461-b4dc-cc4dc8ce3659@tundraware.com> On 06/29/2017 09:03 AM, Grant Edwards wrote: > I've forked a copy of https://github.com/Roguelazer/muttdown and have > been adding a few features and fixing a few bugs. It's meant to be When doing this sort of thing, I find 'pew' virtual environments immensely helpful. They allow you to control python configs and pip loads in userland without having to fiddle with the system versions of things. Highly recommended: https://github.com/berdario/pew From info at tundraware.com Thu Jun 29 11:39:13 2017 From: info at tundraware.com (Tim Daneliuk) Date: Thu, 29 Jun 2017 10:39:13 -0500 Subject: Development testing without reinstalling egg constantly? In-Reply-To: References: Message-ID: <1ijf2e-dl81.ln1@oceanview.tundraware.com> On 06/29/2017 09:03 AM, Grant Edwards wrote: > I've forked a copy of https://github.com/Roguelazer/muttdown and have > been adding a few features and fixing a few bugs. It's meant to be When doing this sort of thing, I find 'pew' virtual environments immensely helpful. They allow you to control python configs and pip loads in userland without having to fiddle with the system versions of things. Highly recommended: https://github.com/berdario/pew From rgaddi at highlandtechnology.invalid Thu Jun 29 12:40:29 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Thu, 29 Jun 2017 09:40:29 -0700 Subject: Development testing without reinstalling egg constantly? In-Reply-To: References: Message-ID: On 06/29/2017 08:32 AM, Grant Edwards wrote: > > This is somebody else's project I'm hacking on, and my understanding > of setup tools is pretty much at the "cargo cult" level. > I have a sneaking suspicion that's everyone. Setuptools (and distutils before it) has no underlying rhyme or reason; just a collection of stuff and if you do all the stuff then magic. Eggs and wheels and twine make me feel like I'm playing the world's worst Settlers of Catan knockoff every time I try to make a project all pretty, and the XKCD 927 factor is in full and glorious effect. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From eryksun at gmail.com Thu Jun 29 13:00:57 2017 From: eryksun at gmail.com (eryk sun) Date: Thu, 29 Jun 2017 17:00:57 +0000 Subject: sys.exc_info In-Reply-To: <5954a32b$0$21718$c3e8da3@news.astraweb.com> References: <5954a32b$0$21718$c3e8da3@news.astraweb.com> Message-ID: On Thu, Jun 29, 2017 at 6:50 AM, Steven D'Aprano wrote: > try: > something > except: > exc_type, exc, tb = sys.exc_info() > print(traceback.extract_tb(tb)) > raise > > Why does it return the exception type separately from the exception, when > the type can be derived by calling `type(exc)`? I think normally it is redundant. While the type and value do have to be tracked separately for the thread's hot exception state (i.e. curexc_type, curexc_value, curexc_traceback), sys.exc_info() returns a caught exception (i.e. exc_type, exc_value, exc_traceback), which is normalized by PyErr_NormalizeException. In CPython you can trivially force the exception value to be None by calling PyErr_SetExcInfo. For example: import sys import ctypes PyErr_SetExcInfo = ctypes.pythonapi.PyErr_SetExcInfo PyErr_SetExcInfo.restype = None PyErr_SetExcInfo.argtypes = (ctypes.py_object,) * 3 PyErr_SetExcInfo(TypeError, None, None) >>> sys.exc_info() (, None, None) Of course this is a silly example, because C. From grant.b.edwards at gmail.com Thu Jun 29 13:01:24 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 29 Jun 2017 17:01:24 +0000 (UTC) Subject: Development testing without reinstalling egg constantly? References: Message-ID: On 2017-06-29, Rob Gaddi wrote: > On 06/29/2017 08:32 AM, Grant Edwards wrote: >> >> This is somebody else's project I'm hacking on, and my understanding >> of setup tools is pretty much at the "cargo cult" level. > > I have a sneaking suspicion that's everyone. Setuptools (and distutils > before it) has no underlying rhyme or reason; just a collection of stuff > and if you do all the stuff then magic. Ah, so it's like git. :) https://xkcd.com/1597/ -- Grant Edwards grant.b.edwards Yow! Let me do my TRIBUTE at to FISHNET STOCKINGS ... gmail.com From tjol at tjol.eu Thu Jun 29 14:19:28 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 29 Jun 2017 20:19:28 +0200 Subject: sys.exc_info In-Reply-To: <5954a32b$0$21718$c3e8da3@news.astraweb.com> References: <5954a32b$0$21718$c3e8da3@news.astraweb.com> Message-ID: <5955456f$0$1738$e4fe514c@news.kpn.nl> On 29/06/17 08:50, Steven D'Aprano wrote: > sys.exc_info() returns three items: > > (exception type, exception value, traceback) > > https://docs.python.org/2/library/sys.html#sys.exc_info > > https://docs.python.org/3/library/sys.html#sys.exc_info > > > > and may be used something like this example: > > > try: > something > except: > exc_type, exc, tb = sys.exc_info() > print(traceback.extract_tb(tb)) > raise > > > > Why does it return the exception type separately from the exception, when > the type can be derived by calling `type(exc)`? > > Ah, Python history. Back in the old days, it was possible to raise strings instead of the classes that took over later. Python 2.4.6 (#1, Jun 29 2017, 19:23:06) [GCC 5.4.0 20160609] on linux4 Type "help", "copyright", "credits" or "license" for more information. >>> try: ... raise 'error', 'message' ... except: ... import sys ... print sys.exc_info() ... ('error', 'message', ) >>> This was deprecated in 2.5, and (I think?) removed in 2.7. >From looking at the old documentation, I have a suspicion that until Python 1.4 (or even earlier than that) the exception "value" was actually never an exception instance. (I'm afraid I can't find a way to install 1.4 and test it on my computer. At least not a reasonable one.) Thomas From python at mrabarnett.plus.com Thu Jun 29 15:00:14 2017 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 29 Jun 2017 20:00:14 +0100 Subject: sys.exc_info In-Reply-To: <5955456f$0$1738$e4fe514c@news.kpn.nl> References: <5954a32b$0$21718$c3e8da3@news.astraweb.com> <5955456f$0$1738$e4fe514c@news.kpn.nl> Message-ID: <6ce5ddae-95df-2345-a9fe-494b62264ceb@mrabarnett.plus.com> On 2017-06-29 19:19, Thomas Jollans wrote: [snip] > > Ah, Python history. > > Back in the old days, it was possible to raise strings instead of the > classes that took over later. > > Python 2.4.6 (#1, Jun 29 2017, 19:23:06) > [GCC 5.4.0 20160609] on linux4 > Type "help", "copyright", "credits" or "license" for more information. >>>> try: > ... raise 'error', 'message' > ... except: > ... import sys > ... print sys.exc_info() > ... > ('error', 'message', ) >>>> > > This was deprecated in 2.5, and (I think?) removed in 2.7. > Deprecated in 2.5, gone in 2.6. >>From looking at the old documentation, I have a suspicion that until > Python 1.4 (or even earlier than that) the exception "value" was > actually never an exception instance. (I'm afraid I can't find a way to > install 1.4 and test it on my computer. At least not a reasonable one.) > From tjol at tjol.eu Thu Jun 29 15:07:03 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 29 Jun 2017 21:07:03 +0200 Subject: sys.exc_info In-Reply-To: References: <5954a32b$0$21718$c3e8da3@news.astraweb.com> Message-ID: <59555096$0$1796$e4fe514c@news.kpn.nl> On 29/06/17 19:00, eryk sun wrote: > On Thu, Jun 29, 2017 at 6:50 AM, Steven D'Aprano wrote: >> try: >> something >> except: >> exc_type, exc, tb = sys.exc_info() >> print(traceback.extract_tb(tb)) >> raise >> >> Why does it return the exception type separately from the exception, when >> the type can be derived by calling `type(exc)`? > > I think normally it is redundant. While the type and value do have to > be tracked separately for the thread's hot exception state (i.e. > curexc_type, curexc_value, curexc_traceback), sys.exc_info() returns a > caught exception (i.e. exc_type, exc_value, exc_traceback), which is > normalized by PyErr_NormalizeException. > > In CPython you can trivially force the exception value to be None by > calling PyErr_SetExcInfo. For example: > > import sys > import ctypes > > PyErr_SetExcInfo = ctypes.pythonapi.PyErr_SetExcInfo > PyErr_SetExcInfo.restype = None > PyErr_SetExcInfo.argtypes = (ctypes.py_object,) * 3 > > PyErr_SetExcInfo(TypeError, None, None) > > >>> sys.exc_info() > (, None, None) > > Of course this is a silly example, because C. > This is doubly silly, as since 1.5 the stdlib documentation has stated that the value "is always a class instance if the exception type is a class object" (... the formulation was changed in py3) From Irv at furrypants.com Thu Jun 29 16:57:42 2017 From: Irv at furrypants.com (Irv Kalb) Date: Thu, 29 Jun 2017 13:57:42 -0700 Subject: Teaching the "range" function in Python 3 Message-ID: I teach Python at two colleges in Silicon Valley. I currently teach an introductory course on Python and most of my students have no programming background whatsoever. Up until now, my colleges have been using Python 2. But now, one of the colleges has made the jump to Python 3. So I am updating my curriculum to Python 3's syntax and concepts. In general, it's all going fine, the changes from raw_input to input, and print from a keyword/statement to the print function have been very straight-forward. But ... Now I am looking at the change in the range function. I completely understand the differences between, and the reasons for, how range works differently in Python 2 vs Python 3. The problem that I've run into has to do with how to explain what range does in Python 3. In Python 2, I easily demonstrated the range function using a simple print statement: print range(0, 10) I discussed how range returns a list. I gave many examples of different values being passed into range, and printing the resulting lists. Next, I introduced the concept of a for loop and show how you use it to iterate over a list, for example: for number in [12, 93, -45.5, 90]: # Some operation using each number (for example, adding up al the numbers) When that was clear, I would go on to explain how you could incorporate range in a for loop: for someVariable in range(0, 10): explaining that range would return a list, and the for statement iterated over that list. Very clear, very easy for new students to understand. But Python 3's version of the range function has been turned into a generator. Again, I understand why this happened, and I agree that this is a good change. The problem is, how can I explain this concept to students who are just learning lists, function and function calls, etc. The concept of how a generator works would be extremely difficult for them to get. The best approach I have seen so far has been to build a simple for loop where I print out the results, like this: for someVariable in range(0, 10): print(someVariable) However, I'm worried that this might lose a lot of students. From their point of view, it might seem difficult to understand that range, which looks like a simple function call passing in the same values each time, returns a different value every time it is called. I am wondering if other teachers have run into this. Is this a real problem? If so, is there any other way of explaining the concept without getting into the underlying details of how a generator works? Do you think it would be helpful to use the words "sequence of numbers" rather than talking about a list here - or would that be even more confusing to students? Thanks, Irv From rosuav at gmail.com Thu Jun 29 17:21:53 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 30 Jun 2017 07:21:53 +1000 Subject: Teaching the "range" function in Python 3 In-Reply-To: References: Message-ID: On Fri, Jun 30, 2017 at 6:57 AM, Irv Kalb wrote: > I am wondering if other teachers have run into this. Is this a real problem? If so, is there any other way of explaining the concept without getting into the underlying details of how a generator works? Do you think it would be helpful to use the words "sequence of numbers" rather than talking about a list here - or would that be even more confusing to students? > The easiest way is to use the word "collection" for things you iterate over (rather than the concrete term "list"). So you can loop through (or iterate over) a collection of explicitly given numbers: for number in [12, 93, -45.5, 90]: or you can loop through a range of numbers: for number in range(10): or you can loop through any number of other things: for file in os.scandir("."): They're all collections. I'd also use this as a great opportunity to talk about naming conventions - a collection is named in the plural ("things") whereas individual items are named in the singular ("thing") - which means that it's very common to have a loop that looks like this: for thing in things: for file in files: for num in numbers: for key in keyring: That rule should help people keep things straight, without being bothered by the difference between lists, ranges, and dictionary views. ChrisA From ian.g.kelly at gmail.com Thu Jun 29 17:43:18 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 29 Jun 2017 15:43:18 -0600 Subject: Teaching the "range" function in Python 3 In-Reply-To: References: Message-ID: On Thu, Jun 29, 2017 at 2:57 PM, Irv Kalb wrote: > Now I am looking at the change in the range function. I completely understand the differences between, and the reasons for, how range works differently in Python 2 vs Python 3. The problem that I've run into has to do with how to explain what range does in Python 3. In Python 2, I easily demonstrated the range function using a simple print statement: > > print range(0, 10) > > I discussed how range returns a list. I gave many examples of different values being passed into range, and printing the resulting lists. In Python 3 you could demonstrate this with: >>> print(list(range(0, 10))) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] > Next, I introduced the concept of a for loop and show how you use it to iterate over a list, for example: > > for number in [12, 93, -45.5, 90]: > # Some operation using each number (for example, adding up al the numbers) > > When that was clear, I would go on to explain how you could incorporate range in a for loop: > > for someVariable in range(0, 10): > > explaining that range would return a list, and the for statement iterated over that list. Very clear, very easy for new students to understand. You can still use the rest of this as is. range no longer returns a list directly but you can demonstrate its contents as a list above, and explain that the for statement iterates over the contents of the range. > But Python 3's version of the range function has been turned into a generator. It's not a generator, although that's a common misunderstanding since range is primarily used for iteration. It's still a sequence type; it just takes advantage of the regularity of its contents to store them efficiently by calculating them on demand rather than keeping them all in one big list. For example, you can still do all this: >>> rng = range(10) >>> print(rng) range(0, 10) >>> len(rng) 10 >>> rng[4] 4 >>> rng[3:5] range(3, 5) >>> rng.index(7) 7 >>> rng.count(4) 1 >>> print(list(reversed(rng[3:5]))) [4, 3] The biggest changes as far as usage goes are that its repr() no longer looks like a list, it's now immutable, and it can't be multiplied or added to other lists -- all of which are resolvable by first converting it to a list. So I don't think that you should really need to alter your approach much to teaching this. From cs at zip.com.au Thu Jun 29 19:14:20 2017 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 30 Jun 2017 09:14:20 +1000 Subject: Teaching the "range" function in Python 3 In-Reply-To: References: Message-ID: <20170629231420.GA43259@cskk.homeip.net> On 29Jun2017 13:57, Irv Kalb wrote: >Now I am looking at the change in the range function. I completely understand >the differences between, and the reasons for, how range works differently in >Python 2 vs Python 3. The problem that I've run into has to do with how to >explain what range does in Python 3. In Python 2, I easily demonstrated the >range function using a simple print statement: > >print range(0, 10) > >I discussed how range returns a list. I gave many examples of different values being passed into range, and printing the resulting lists. [... and the teaching path from there ...] > >But Python 3's version of the range function has been turned into a generator. Again, I understand why this happened, and I agree that this is a good change. The problem is, how can I explain this concept to students who are just learning lists, function and function calls, etc. The concept of how a generator works would be extremely difficult for them to get. The best approach I have seen so far has been to build a simple for loop where I print out the results, like this: > >for someVariable in range(0, 10): > print(someVariable) [...] How feasible is it to try a functional approach (in the "functional programming" sense)? They already have lists I understand. Explain that a list is a concrete explicit expression of a sequence of values. Then that there are other ways to express a sequence of values (and allude that there are many ways to express various things). So that when one speaks the prose "the values from A to B", that implies the values A, A+1, etc up (to B-1 in python). In many cases it doesn't matter wther you actaully compute them unless you need a specific value. So "range(A, B)" is a Python builtin expressing the prose "the values from A to B". It doesn't compute them until you need the specific values. This makes it small (no huge list of things for a huge range) and fast (no need to actually count out all the values). Then show them that you can still use it in iteration situations, with the for loop being the classic example. As a bonus, later you can use this paradigm when talking about generators and classes that implement "large but only computed on demand" things. Cheers, Cameron Simpson From greg.ewing at canterbury.ac.nz Thu Jun 29 20:22:29 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 30 Jun 2017 12:22:29 +1200 Subject: Teaching the "range" function in Python 3 In-Reply-To: References: Message-ID: Irv Kalb wrote: > In Python 2, I easily > demonstrated the range function using a simple print statement: > > print range(0, 10) > > I discussed how range returns a list. I gave many examples of different > values being passed into range, and printing the resulting lists. > > Next, I introduced the concept of a for loop and show how you use it to > iterate over a list, for example: > > for number in [12, 93, -45.5, 90]: # Some operation using each number (for > example, adding up al the numbers) > > When that was clear, I would go on to explain how you could incorporate range > in a for loop: > > for someVariable in range(0, 10): Don't start with range(). Start with lists, and introduce the for loop as a way to iterate over lists. Leave range() until much later. You should be able to go a *long* way without it -- it's quite rare to need to iterate over a range of ints in idiomatic Python code. When you do get to range(), just say it returns an object that produces a series of ints when you iterate over it. By now they should have seen enough examples of other iterable objects -- lists, tuples, etc. -- that this will make sense. DON'T call it a "generator", btw -- that term is reserved for a very special kind of iterator, and range() isn't one of them. -- Greg From rantingrickjohnson at gmail.com Thu Jun 29 22:33:37 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Thu, 29 Jun 2017 19:33:37 -0700 (PDT) Subject: Teaching the "range" function in Python 3 In-Reply-To: References: Message-ID: On Thursday, June 29, 2017 at 4:01:07 PM UTC-5, Irv Kalb wrote: > > [...] > > But Python 3's version of the range function has been > turned into a generator. Again, I understand why this > happened, and I agree that this is a good change. The > problem is, how can I explain this concept to students who > are just learning lists, function and function calls, etc. > The concept of how a generator works would be extremely > difficult for them to get. The best approach I have seen > so far has been to build a simple for loop where I print > out the results, like this: > > for someVariable in range(0, 10): > print(someVariable) > > However, I'm worried that this might lose a lot of > students. From their point of view, it might seem > difficult to understand that range, which looks like a > simple function call passing in the same values each time, > returns a different value every time it is called. You are making the same fatal mistake that many "too bright for their own good" teachers have made since antiquity, and that is, you are bombarding the student with too many concepts at once. And for the archetypal pythonic example of what *NOT* to do, one need look no further than the grease and semen stained pages of the official python tutorial: [QUOTE] 4.6. Defining Functions >>> def fib(n): # write Fibonacci series up to n ... """Print a Fibonacci series up to n.""" ... a, b = 0, 1 ... while a < n: ... print(a, end=' ') ... a, b = b, a+b ... print() ... >>> # Now call the function we just defined: ... fib(2000) 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 [/QUOTE] As you can see, the author seems more interested in showboating than teaching. :-( If the purpose of this section is to teach the fundamentals of of functions, then the author has failed miserably. A better *FIRST* example would be something like this: def add(x, y): return x + y When teaching a student about functions, the first step is to help them understand *WHY* they need to use functions, and the second is to teach them how to define a function. In my simplistic example, the practical necessity of functions can be easily intuited by the student without the distractions of (1) doc-strings, (2) tuple unpacking, (3) an inner loop structure, (4) implicitly writing to IO streams using the print function, (5) using advanced features of the print function, (6) more tuple unpacking (this time with a twist of lime), (7) and an implicit newline insertion using the print function (deep breath) Now, would someone please remind me, again, what were we learning about??? GHEEZ! So with that plate of scholastic spaghetti in mind, let's return to your dilemma: > for someVariable in range(0, 10): > print(someVariable) > > However, I'm worried that this might lose a lot of > students. From their point of view, it might seem > difficult to understand that range, which looks like a > simple function call passing in the same values each time, > returns a different value every time it is called. First of all, the range function is only called *ONCE* in your example, which can easily be demonstrated by executing this code... >>> for value in range(10): ... print(value) ... 0 1 2 3 4 5 6 7 8 9 And to further drive home the point, you can manually insert a list literal to prove this: >>> range(10) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> for value in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]: ... print(value) ... 0 1 2 3 4 5 6 7 8 9 Furthermore, you should not introduce loops and the range function in the same example. Do not follow the lead of the Python tutorial author. Instead, introduce both concepts separately, and using the most simple example you can muster. >>> for char in "abc": ... print(char) ... a b c Followed by... >>> for item in (1,2,3): ... print(item) ... 1 2 3 Now, once the student understands that range simply returns a list of things, and that "for loops" iterate over each _thing_ in a collection of _things_ (be that collection a list, or a tuple, or a string, or whatever...), the student will not be so dumbfounded by this code >>> for value in range(10): ... print(value) ... 0 1 2 3 4 5 6 7 8 9 > I am wondering if other teachers have run into this. Is > this a real problem? If so, is there any other way of > explaining the concept without getting into the underlying > details of how a generator works? Generators are an advanced topic and should not be presented until the student has a firm grasp of general programming fundamentals. In python, the general fundamentals would be a very basic, but confident, understanding of: integers, floats, strings, lists, tuples, dicts, variables, conditionals, loops, and functions. Followed by an in depth study of the aforementioned. > Do you think it would be helpful to use the words "sequence > of numbers" rather than talking about a list here - or > would that be even more confusing to students? Being that the word "sequence" is highly generalized and naturally intuitive, i don't find it to be confusing at all. OTOH, the word "list" falls more into the realms of defining organizational behavoirs that are specific to humans and written language. But hey, tell that to the idiot who designed virtual strings with escape codes for such distinct mechanical phenomenon as line-feeds and carriage-returns! From rosuav at gmail.com Thu Jun 29 22:57:47 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 30 Jun 2017 12:57:47 +1000 Subject: Teaching the "range" function in Python 3 In-Reply-To: References: Message-ID: On Fri, Jun 30, 2017 at 12:33 PM, Rick Johnson wrote: > A better *FIRST* example would be > something like this: > > def add(x, y): > return x + y > > When teaching a student about functions, the first step is > to help them understand *WHY* they need to use functions, > and the second is to teach them how to define a function. In > my simplistic example, the practical necessity of functions > can be easily intuited by the student without the > distractions... ... except that you've made your first function so trivial that it doesn't need to be a function. How does that help anyone understand why they need functions? What's the point of writing "add(5, 7)" rather than "5 + 7"? So you need something else around the outside to justify even having a function at all. In the example in the tutorial, the details ... > ... of (1) doc-strings, (2) tuple unpacking, (3) an > inner loop structure, (4) implicitly writing to IO streams > using the print function, (5) using advanced features of the > print function, (6) more tuple unpacking (this time with a > twist of lime), (7) and an implicit newline insertion using > the print function don't need to be explained at all! They *just work*. You also don't have to explain in detail how garbage collection works, the way that Python's object and type models work, or how the electrons in your CPU are kept from portalling across in a quantum tunnel. None of that matters. So I support the tutorial's example over yours. ChrisA From rantingrickjohnson at gmail.com Thu Jun 29 23:03:06 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Thu, 29 Jun 2017 20:03:06 -0700 (PDT) Subject: Converting sentences to vector form In-Reply-To: <9207d59d-be07-4ab1-878b-3c37cabebbea@googlegroups.com> References: <06e7ad25-a1d2-4925-8066-8f644e6b5beb@googlegroups.com> <87vanh1yf3.fsf@bsb.me.uk> <9207d59d-be07-4ab1-878b-3c37cabebbea@googlegroups.com> Message-ID: <6100c86c-815a-45dc-b015-3e9e81742501@googlegroups.com> On Tuesday, June 27, 2017 at 6:15:31 AM UTC-5, Bhaskar Dhariyal wrote: > You can't train a model on words. https://youtu.be/dSIKBliboIo?t=52 From chaneybenjamini at gmail.com Fri Jun 30 00:06:23 2017 From: chaneybenjamini at gmail.com (Benjamin Chaney) Date: Fri, 30 Jun 2017 00:06:23 -0400 Subject: import queue in Python 2 and Python 3 Message-ID: What is the best way to import the synchronized queue class that is compatible with both Python2 and Python3. Right now I am using the following: >if sys.version_info < (3, 0): > import Queue as queue >else: > import queue This seems somewhat unpythonic. Is there a better way without installing extra packages? Thanks, Ben From rosuav at gmail.com Fri Jun 30 00:37:59 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 30 Jun 2017 14:37:59 +1000 Subject: import queue in Python 2 and Python 3 In-Reply-To: References: Message-ID: On Fri, Jun 30, 2017 at 2:06 PM, Benjamin Chaney wrote: > What is the best way to import the synchronized queue class that is > compatible with both Python2 and Python3. Right now I am using the > following: > >>if sys.version_info < (3, 0): >> import Queue as queue >>else: >> import queue > > This seems somewhat unpythonic. Is there a better way without > installing extra packages? > try: import queue except ImportError: # Python 2 import Queue as queue ChrisA From rustompmody at gmail.com Fri Jun 30 00:44:00 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 29 Jun 2017 21:44:00 -0700 (PDT) Subject: Teaching the "range" function in Python 3 In-Reply-To: References: Message-ID: On Friday, June 30, 2017 at 8:28:23 AM UTC+5:30, Chris Angelico wrote: > On Fri, Jun 30, 2017 at 12:33 PM, Rick Johnson wrote: > > A better *FIRST* example would be > > something like this: > > > > def add(x, y): > > return x + y > > > > When teaching a student about functions, the first step is > > to help them understand *WHY* they need to use functions, > > and the second is to teach them how to define a function. In > > my simplistic example, the practical necessity of functions > > can be easily intuited by the student without the > > distractions... > > ... except that you've made your first function so trivial that it > doesn't need to be a function. How does that help anyone understand > why they need functions? What's the point of writing "add(5, 7)" > rather than "5 + 7"? So you need something else around the outside to > justify even having a function at all. Same logic applies to fibonacci also ? ?Why the f___ are these old fogies teaching us fibonacci and this other crap instead of IOT/Big-Data/how-to-write-an-app/cloud?? or whatever else is hot And if you think this is a theoretical strawman it just means youve not taught enough to know that motivation is at least big a problem as any specific technique/algorithm/language/technology/etc So yes Rick's example has slightly less motivation than fib But if you are focussing on technique its better than the fib ? at least it does not contain a blithering print ? a point I am sure you will agree with Chris From __peter__ at web.de Fri Jun 30 02:22:42 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 30 Jun 2017 08:22:42 +0200 Subject: Teaching the "range" function in Python 3 References: Message-ID: Gregory Ewing wrote: > Don't start with range(). Start with lists, and introduce the for > loop as a way to iterate over lists. Leave range() until much later. > You should be able to go a *long* way without it -- it's quite > rare to need to iterate over a range of ints in idiomatic Python > code. Indeed. Among beginners the for i in range(len(items)): print(items[i]) idiom is already too common. From devilsgrin94 at gmail.com Fri Jun 30 03:51:43 2017 From: devilsgrin94 at gmail.com (devilsgrin94 at gmail.com) Date: Fri, 30 Jun 2017 00:51:43 -0700 (PDT) Subject: geckodriver not working while using Selenium Message-ID: #I am using the code below to set preferences so I can use Selenium. import os from selenium import webdriver fp = webdriver.FirefoxProfile() fp.set_preference("browser.download.folderList",2) fp.set_preference("browser.download.manager.showWhenStarting",False) fp.set_preference("browser.download.dir", os.getcwd()) fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/PDF") fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/Document") fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/File") fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/PDF") fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/Archive") #After the last live mentioned above I am getting error in the next line driver = webdriver.Firefox(firefox_profile=fp) driver.get("https://www.facebook.com/groups/kolkataitservices/files/") #I get this error #Traceback (most recent call last): # File "easydpy.py", line 15, in # driver = webdriver.Firefox(firefox_profile=fp) # File "E:\Python27\lib\site-#packages\selenium\webdriver\firefox\webdriver.py", line 142, in __init__ # self.service.start() # File "E:\Python27\lib\site-packages\selenium\webdriver\common\service.py", #line 81, in start # os.path.basename(self.path), self.start_error_message) #selenium.common.exceptions.WebDriverException: Message: 'geckodriver' #executable needs to be in PATH. #Please help me resolve this issue. From sailor at lists.xtsubasa.org Fri Jun 30 03:53:17 2017 From: sailor at lists.xtsubasa.org (Pavel Volkov) Date: Fri, 30 Jun 2017 10:53:17 +0300 Subject: urllib.request with proxy and HTTPS Message-ID: <00091eee-ce28-4ad0-84e4-8dc36c6cdfe5@lists.xtsubasa.org> Hello, I'm trying to make an HTTPS request with urllib. OS: Gentoo Python: 3.6.1 openssl: 1.0.2l This is my test code: ===== CODE BLOCK BEGIN ===== import ssl import urllib.request from lxml import etree PROXY = 'proxy.vpn.local:9999' URL = "https://google.com" proxy = urllib.request.ProxyHandler({'http': PROXY}) #context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_1) context = ssl.SSLContext() context.verify_mode = ssl.CERT_REQUIRED context.check_hostname = True secure_handler = urllib.request.HTTPSHandler(context = context) opener = urllib.request.build_opener(proxy, secure_handler) opener.addheaders = [('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0')] response = opener.open(URL) tree = etree.parse(response, parser=etree.HTMLParser()) print(tree.docinfo.doctype) ===== CODE BLOCK END ===== My first problem is that CERTIFICATE_VERIFY_FAILED error happens. I've found that something similar happens in macOS since Python installs its own set of trusted CA. But this isn't macOS and I can fetch HTTPS normally with curl and other tools. ===== TRACE BLOCK BEGIN ===== Traceback (most recent call last): File "/usr/lib64/python3.6/urllib/request.py", line 1318, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "/usr/lib64/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib64/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib64/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib64/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/usr/lib64/python3.6/http/client.py", line 964, in send self.connect() File "/usr/lib64/python3.6/http/client.py", line 1400, in connect server_hostname=server_hostname) File "/usr/lib64/python3.6/ssl.py", line 401, in wrap_socket _context=self, _session=session) File "/usr/lib64/python3.6/ssl.py", line 808, in __init__ self.do_handshake() File "/usr/lib64/python3.6/ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "/usr/lib64/python3.6/ssl.py", line 683, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./https_test.py", line 21, in response = opener.open(URL) File "/usr/lib64/python3.6/urllib/request.py", line 526, in open response = self._open(req, data) File "/usr/lib64/python3.6/urllib/request.py", line 544, in _open '_open', req) File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/usr/lib64/python3.6/urllib/request.py", line 1361, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib64/python3.6/urllib/request.py", line 1320, in do_open raise URLError(err) urllib.error.URLError: ===== TRACE BLOCK END ===== Second problem is that for HTTP requests proxy is used, but for HTTPS it makes a direct connection (verified with tcpdump). I've read at docs.python.org that previous versions of Python couldn't handle HTTPS with proxy but that shortcoming seems to have gone now. Please help :) From tjol at tjol.eu Fri Jun 30 04:34:43 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 30 Jun 2017 10:34:43 +0200 Subject: import queue in Python 2 and Python 3 In-Reply-To: References: Message-ID: <59560d30$0$1822$e4fe514c@news.kpn.nl> On 30/06/17 06:06, Benjamin Chaney wrote: > What is the best way to import the synchronized queue class that is > compatible with both Python2 and Python3. Right now I am using the > following: > >> if sys.version_info < (3, 0): >> import Queue as queue >> else: >> import queue > > This seems somewhat unpythonic. Is there a better way without > installing extra packages? What Chris said. If you find yourself doing a lot of this kind of thing, you might want to use six, though, and do from six.moves import queue From jobmattcon at gmail.com Fri Jun 30 06:35:41 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Fri, 30 Jun 2017 03:35:41 -0700 (PDT) Subject: how to add new tuple as key in dictionary? Message-ID: <344c23f8-f440-4c23-a5e6-9f93b0145e02@googlegroups.com> I find that list can not be key in dictionary then find tuple can be as key but when I add new tuple as key , got error in python 2.7 groupkey = {(0,0): []} groupkey[tuple([0,3])] = groupkey[tuple([0,3])] + [[0,1]] From __peter__ at web.de Fri Jun 30 07:02:04 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 30 Jun 2017 13:02:04 +0200 Subject: how to add new tuple as key in dictionary? References: <344c23f8-f440-4c23-a5e6-9f93b0145e02@googlegroups.com> Message-ID: Ho Yeung Lee wrote: > I find that list can not be key in dictionary > then find tuple can be as key > > but when I add new tuple as key , got error in python 2.7 > > groupkey = {(0,0): []} > groupkey[tuple([0,3])] = groupkey[tuple([0,3])] + [[0,1]] First try to understand that you get the same error with every non-existent key: >>> groupkey = {0: []} >>> groupkey[1] = groupkey[1] + [2] Traceback (most recent call last): File "", line 1, in KeyError: 1 Second, it looks like you want to change a list as the value, assuming the empty list for non-existent keys. The straight-forward way to do this is: >>> pairs = [ ... ((0, 0), "foo"), ... ((0, 1), "bar"), ... ((0, 0), "baz"), ... ] >>> groups = {} >>> for k, v in pairs: ... if k in groups: ... groups[k].append(v) ... else: ... groups[k] = [v] ... >>> groups {(0, 1): ['bar'], (0, 0): ['foo', 'baz']} You can simplify this with the setdefault() method which in the example below will add an empty list for non-existent keys: >>> groups = {} >>> for k, v in pairs: groups.setdefault(k, []).append(v) ... >>> groups {(0, 1): ['bar'], (0, 0): ['foo', 'baz']} Finally there's a dedicated class for your use case: >>> from collections import defaultdict >>> groups = defaultdict(list) >>> for k, v in pairs: ... groups[k].append(v) ... >>> groups defaultdict(, {(0, 1): ['bar'], (0, 0): ['foo', 'baz']}) Missing entries spring into existence automatically -- defaultdict creates the value by calling the function (list in this case) passed to the constructor. From stephen_tucker at sil.org Fri Jun 30 07:04:42 2017 From: stephen_tucker at sil.org (Stephen Tucker) Date: Fri, 30 Jun 2017 12:04:42 +0100 Subject: how to add new tuple as key in dictionary? In-Reply-To: <344c23f8-f440-4c23-a5e6-9f93b0145e02@googlegroups.com> References: <344c23f8-f440-4c23-a5e6-9f93b0145e02@googlegroups.com> Message-ID: Hello there, Ho Yeung Lee, The use of groupkey[tuple([0,3])] to the right of the = is attempting to access an entry in your dictionary that you have not already created. That is why you are getting the error message. You need to create an entry whose key is tuple([0,3]) before you can access it. The following code does not provoke an error message: groupkey = {(0,0): []} groupkey = {tuple([0,3]): []} groupkey[tuple([0,3])] = groupkey[tuple([0,3])] + [[0,1]] I hope this helps you. Stephen. Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Fri, Jun 30, 2017 at 11:35 AM, Ho Yeung Lee wrote: > I find that list can not be key in dictionary > then find tuple can be as key > > but when I add new tuple as key , got error in python 2.7 > > groupkey = {(0,0): []} > groupkey[tuple([0,3])] = groupkey[tuple([0,3])] + [[0,1]] > -- > https://mail.python.org/mailman/listinfo/python-list > From alt.nl-5yfi4h8 at yopmail.com Fri Jun 30 07:29:00 2017 From: alt.nl-5yfi4h8 at yopmail.com (aa) Date: Fri, 30 Jun 2017 11:29:00 +0000 Subject: geckodriver not working while using Selenium In-Reply-To: References: Message-ID: gecko ? From pavol.lisy at gmail.com Fri Jun 30 07:32:04 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Fri, 30 Jun 2017 13:32:04 +0200 Subject: DJANGO cannot import name _compare_digest In-Reply-To: <742121c3-4a51-4091-802a-8e7d65a0979d@googlegroups.com> References: <78efb412-7b8f-4aea-926f-e4f418805c45@googlegroups.com> <742121c3-4a51-4091-802a-8e7d65a0979d@googlegroups.com> Message-ID: On 6/28/17, Xristos Xristoou wrote: > i dont have 'operator.py' in my system only 'test_operator.py' and > 'fix_operator.py' > > if : > import sys > print sys.modules['operator'] > > i get this : > > that say me > > how to rename it? > -- > https://mail.python.org/mailman/listinfo/python-list Something interesting here! I just tried to simulate your problems: python 2.7.12 >>> import logging as operator # I want to mask operator module >>> from operator import _compare_digest as compare_digest # WHAT? >>> operator.__name__ # OK 'logging' >>> operator._compare_digest # OK Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute '_compare_digest' >>> compare_digest.__name__ # ? '_compare_digest' >>> compare_digest.__class__ # ? >>> compare_digest.__module__ # ? 'operator' >>> operator.log.__module__ # OK 'logging' python 3.6.1 works as I expected >>> import logging as operator >>> from operator import _compare_digest as compare_digest Traceback (most recent call last): File "", line 1, in ImportError: cannot import name '_compare_digest' From Rasputin at Leningrad.Ru Fri Jun 30 08:29:00 2017 From: Rasputin at Leningrad.Ru (Rasputin) Date: Fri, 30 Jun 2017 12:29:00 +0000 Subject: urllib In-Reply-To: References: <00091eee-ce28-4ad0-84e4-8dc36c6cdfe5@lists.xtsubasa.org> Message-ID: good luck with that, mate ! From Rasputin at Leningrad.Ru Fri Jun 30 08:35:00 2017 From: Rasputin at Leningrad.Ru (Rasputin) Date: Fri, 30 Jun 2017 12:35:00 +0000 Subject: argparse epilog call function? In-Reply-To: <5a66c164-beac-470e-9bec-779d2abce9e0@googlegroups.com> References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> <6d78f59d-fd7d-4144-b170-1711297a7d0b@googlegroups.com> <5a66c164-beac-470e-9bec-779d2abce9e0@googlegroups.com> Message-ID: will you ever post a single .py file which actually works ? all the stuff you post is breaking. From Rasputin at Leningrad.Ru Fri Jun 30 08:44:00 2017 From: Rasputin at Leningrad.Ru (Rasputin) Date: Fri, 30 Jun 2017 12:44:00 +0000 Subject: why are these 2 fucking clowns in here ? In-Reply-To: <8354d529-907d-4b69-80b3-a2ebcda15698@googlegroups.com> References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> <8354d529-907d-4b69-80b3-a2ebcda15698@googlegroups.com> Message-ID: C. Angelico and Mark Lawrence, who the fuck does this twin-scum think they are ? you better stfu right now. these guys are responsible for turning the pythonWiki into a pile of rubbish. fuck off and never return ! breamoreboy at gmail.com: > On Tuesday, June 27, 2017 at 3:25:10 PM UTC+1, Chris Angelico wrote: >> On Wed, Jun 28, 2017 at 12:09 AM, Fox wrote: >>> what " -h " are you even talkin bout ? >>> >>> >>> >>> >>> def Examples(): >>> text = """Lots of examples""" >>> print(text.format()) >>> >>> >>> >>> epilog='where the heck to put a -h ?????? ' >>> >>> epilog=Examples() >> >> List admins, this person has been abusing the pydotorg-www list for >> several days. Can he please be banned? >> >> ChrisA > > I'll second that, the wxpython lists have been suffering similar abuse for several days. > > Kindest regards. > > Mark Lawrence From breamoreboy at gmail.com Fri Jun 30 08:55:37 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Fri, 30 Jun 2017 05:55:37 -0700 (PDT) Subject: urllib In-Reply-To: References: <00091eee-ce28-4ad0-84e4-8dc36c6cdfe5@lists.xtsubasa.org> Message-ID: On Friday, June 30, 2017 at 1:30:10 PM UTC+1, Rasputin wrote: > good luck with that, mate ! Please don't change the subject line and also provide some context when you reply, we're not yet mindreaders :) Kindest regards. -- Mark Lawrence. From darcy at VybeNetworks.com Fri Jun 30 08:56:55 2017 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Fri, 30 Jun 2017 08:56:55 -0400 Subject: why are these 2 fucking clowns in here ? In-Reply-To: References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> <8354d529-907d-4b69-80b3-a2ebcda15698@googlegroups.com> Message-ID: <6fd8a634-f3d4-9d71-8bd2-a49fc7d48964@VybeNetworks.com> On 06/30/17 08:44, Rasputin wrote: > fuck off and never return ! Chill dude. Haven't you ever heard of a killfile? You just earned a place in mine. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From breamoreboy at gmail.com Fri Jun 30 08:59:38 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Fri, 30 Jun 2017 05:59:38 -0700 (PDT) Subject: why are these 2 fucking clowns in here ? In-Reply-To: References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> <8354d529-907d-4b69-80b3-a2ebcda15698@googlegroups.com> Message-ID: <2c793ed0-ba29-4ef0-a8fa-8de496265881@googlegroups.com> On Friday, June 30, 2017 at 1:45:00 PM UTC+1, Rasputin wrote: > C. Angelico and Mark Lawrence, > > who the fuck does this twin-scum think they are ? > > you better stfu right now. > > these guys are responsible for turning the pythonWiki into a pile of > rubbish. pythonWiki??? > > fuck off and never return ! > One strike and you're out. *plonk* Mark Lawrence From mmueller at python-academy.de Fri Jun 30 11:19:03 2017 From: mmueller at python-academy.de (=?UTF-8?Q?Mike_M=c3=bcller?=) Date: Fri, 30 Jun 2017 17:19:03 +0200 Subject: PyCon.DE 2017 - Early Bird Deadline July 2, 2017 Message-ID: PyCon.DE 2017 - Early Bird Deadline July 2, 2017 ------------------------------------------------ There are only 2 days left to by a ticket at the earl bird rate for PyCon.DE. https://de.pycon.org/#tickets About PyCon.DE -------------- The next PyCon.DE will be held from 25-27th October 2017 at the ZKM - center for art and media in Karlsruhe, Germany. The conference is the sixth in this series, gathering Python professionals and enthusiasts. The conference language will English, attracting Pythonistas from all over Europe and the world. Volunteers organize and provide talks, tutorials, as well as discussions in an open-source fashion. This Karlsruhe conference has a special focus on data science. Therefore, a PyData conference is held as part of PyCon.DE. PyData is an internationally renowned conference series, which is dedicated to data science, machine learning, and AI. The ZKM | Center for Art and the Media is a unique cultural institution, as it is a place that extends the original tasks of the museum. It is a home of all media and genres, a house of both the traditional arts like painting, photography, and sculpture as well as the time-based arts such as film, video, media art, music, dance, theater, and performance. Call for proposals is still open until 1st of October. Registration for attending the conference will start soon. Don't miss this event! More information available at: http://www.pycon.de http://pydata.org http://zkm.de From mmueller at python-academy.de Fri Jun 30 11:39:55 2017 From: mmueller at python-academy.de (=?UTF-8?Q?Mike_M=c3=bcller?=) Date: Fri, 30 Jun 2017 17:39:55 +0200 Subject: EuroSciPy 2017 - Call for Contributions extended to July 2 Message-ID: Call for Contributions ---------------------- There are only 2 days left to submit a talk, tutorial, or poster for EuroSciPy 2017 (https://www.euroscipy.org/2017/). Don't miss the change to talk about what you do with Python in science. Submit your proposal today: https://www.papercall.io/euroscipy-2017 About EuroSciPy --------------- The EuroSciPy meeting is a cross-disciplinary gathering focused on the use and development of the Python language in scientific research. This event strives to bring together both users and developers of scientific tools, as well as academic research and state of the art industry. Presentations of scientific tools and libraries using the Python language, including but not limited to: Vector and array manipulation Parallel computing Scientific visualization Scientific data flow and persistence Algorithms implemented or exposed in Python Web applications and portals for science and engineering- Reports on the use of Python in scientific achievements or ongoing projects. General-purpose Python tools that can be of special interest to the scientific community. This year, we have announced our two keynote speakers: Julia Roher, one of the authors of the blog the http://www.the100.ci/, who will discuss scientific reproducibility and statistical analysis, and Soumith Chintala https://research.fb.com/people/chintala-soumith/ , Senior Researcher at Facebook AI Research, who will talk about using pytorch for Deep Learning http://pytorch.org/ From steve+python at pearwood.info Fri Jun 30 11:58:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Jul 2017 01:58:56 +1000 Subject: Teaching the "range" function in Python 3 References: Message-ID: <59567542$0$1591$c3e8da3$5496439d@news.astraweb.com> Hello Irv, and welcome! Good to have a teacher of Python here! On Fri, 30 Jun 2017 06:57 am, Irv Kalb wrote: [...] > Now I am looking at the change in the range function. I completely understand > the differences between, and the reasons for, how range works differently in > Python 2 vs Python 3. The problem that I've run into has to do with how to > explain what range does in Python 3. How do you explain what xrange() does in Python 2? range() is the same. I'm not really sure you need to explain what range() does in a lot of detail. The technical details are a little advanced for complete beginners. Although this might make a nice project for moderately advanced students: "Write a class that works like range, that produces a series of numbers 0, 1, 2, 4, 8, 16, 32, ... up to some limit." The secret is the __getitem__ method. > In Python 2, I easily demonstrated the > range function using a simple print statement: > > print range(0, 10) In Python 3, that would be: print(list(range(10))) (no need to give the start value if it is zero). > I discussed how range returns a list. I gave many examples of different > values being passed into range, and printing the resulting lists. range() in Python 3 is just a lazy list. Instead of producing the values all at once, and populating a real list (an array of values), range() produces them only when requested. In Python 2, you have range(10) produce a list, and fill it with 10 values: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] When you iterate over that, the interpreter says: list[0] "Give me the first value" -- retrieves 0 from the list list[1] "Give me the second value" -- retrieves 1 from the list list[2] "Give me the third value" -- retrieves 2 from the list and so forth, until there are no more values. In Python 3, the range object does NOT create a list all at once. Instead, when the interpreter says: list[0] "Give me the first value" the range object (a "lazy list") calculates that the value will be 0, and returns it; when the interpreter says: list[6] "Give me the seventh value" the lazy list calculates the seventh value, namely 6, and returns it. The advantage is not very noticeable for range(10), but consider range(10000000) instead. [...] > But Python 3's version of the range function has been turned into a generator. It actually isn't a generator, and you shouldn't use that terminology. A generator could only be iterated over once. range objects can be iterated over and over again. Generators don't provide random access to any item, range objects do. Generators don't know their length: py> def range_generator(start, end): ... for x in range(start, end): ... yield x ... py> len(range_generator(0, 10)) Traceback (most recent call last): File "", line 1, in TypeError: object of type 'generator' has no len() but range objects do: py> len(range(0, 10)) 10 py> len(range(35, 1077, 2)) 521 In fact, range objects are even sliceable! py> range(35, 1077, 2)[50:200:3] range(135, 435, 6) Talking about that is probably a bit too advanced for beginners, but it does show why you shouldn't refer to range as a generator. The most accurate description would be a lazy sequence, or a sequence where the values are calculated on demand. > Again, I understand why this happened, and I agree that this is a good > change. The problem is, how can I explain this concept to students who are > just learning lists, function and function calls, etc. The concept of how a > generator works would be extremely difficult for them to get. The best > approach I have seen so far has been to build a simple for loop where I print > out the results, like this: > > for someVariable in range(0, 10): > print(someVariable) > > However, I'm worried that this might lose a lot of students. From their point > of view, it might seem difficult to understand that range, which looks like a > simple function call passing in the same values each time, returns a different > value every time it is called. It is important that the students understand that range is not being called each time through the loop. It only gets called once, at the start of the for-loop. One way to understand how it works is to think of it as a while loop: temp = range(10) # only called once, not every time through the loop i = 0 while i < len(temp): someVariable = temp[i] print(someVariable) i += 1 Of course I don't suggest you teach your students to think of for loops in this way -- at least not until after you've taught them while loops. I think it is best to teach them for loops as the primitive operation, and only later tell them that "under the hood" for loops are actually while loops. But the important factor is that there's nothing mysterious going on with the call to random. It returns the same value each time it is called, and it is only called once, at the start of the loop. In some ways, it is just like a list, except instead of pre-populating the list: [0, 1, 2, 3, ... , 10000000] it only calculates the values on request. The for loop tells Python to request those values, one per loop. > I am wondering if other teachers have run into this. Is this a real problem? > If so, is there any other way of explaining the concept without getting into > the underlying details of how a generator works? Do you think it would be > helpful to use the words "sequence of numbers" rather than talking about a > list here - or would that be even more confusing to students? Sequence is definitely the right way to go, even if it is a little more difficult to get across. Lists have a bunch of other methods you can use, like list.sort() and list.reverse(), which range objects don't have. They're not lists, but they do behave a little like them. But ultimately, since you are teaching beginners, the difference between a list of numbers and a sequence of numbers is not very important right at the beginning. Hope I've given you some food for thought! -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From gbs.deadeye at gmail.com Fri Jun 30 12:01:53 2017 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Fri, 30 Jun 2017 16:01:53 +0000 Subject: why are these 2 fucking clowns in here ? In-Reply-To: <6fd8a634-f3d4-9d71-8bd2-a49fc7d48964@VybeNetworks.com> References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> <8354d529-907d-4b69-80b3-a2ebcda15698@googlegroups.com> <6fd8a634-f3d4-9d71-8bd2-a49fc7d48964@VybeNetworks.com> Message-ID: Just don't read it. Calm down. From steve+python at pearwood.info Fri Jun 30 12:03:03 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Jul 2017 02:03:03 +1000 Subject: Teaching the "range" function in Python 3 References: Message-ID: <59567638$0$1591$c3e8da3$5496439d@news.astraweb.com> On Fri, 30 Jun 2017 09:17 am, Stefan Ram wrote: >>>> b = a.__iter__() Don't do that. Dunder ("Double UNDERscore") methods like __iter__ should only be called by the Python interpreter, not by the programmer. The right way to create an iterator is to call the built-in function iter: b = iter(a) The iter() function may call a.__iter__ (technically, it calls type(a).__iter__ instead) but it may also do other things, which you miss out on if you call __iter__ directly. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From info at wingware.com Fri Jun 30 12:05:29 2017 From: info at wingware.com (Wingware) Date: Fri, 30 Jun 2017 12:05:29 -0400 Subject: ANN: Wing Python IDE 6.0.6 released Message-ID: <595676C9.9090509@wingware.com> Hi, We've just released Wing 6.0.6 which further improves remote development, adds preferences to avoid problems seen when debugging odoo and some I/O intensive threaded code, solves some auto-completion and auto-editing problems, fixes a few VI mode bugs, remembers editor zoom level between sessions, and makes about 40 other minor improvements. For details, see http://wingware.com/pub/wingide/6.0.6/CHANGELOG.txt Wing 6 is the latest major release in Wingware's family of Python IDEs, including Wing Pro, Wing Personal, and Wing 101. Wing 6 adds many new features, introduces a new annual license option for Wing Pro, and makes Wing Personal free. New Features in Wing 6 * Improved Multiple Selections: Quickly add selections and edit them all at once * Easy Remote Development: Work seamlessly on remote Linux, OS X, and Raspberry Pi systems * Debugging in the Python Shell: Reach breakpoints and exceptions in (and from) the Python Shell * Recursive Debugging: Debug code invoked in the context of stack frames that are already being debugged * PEP 484 and PEP 526 Type Hinting: Inform Wing's static analysis engine of types it cannot infer * Support for Python 3.6 and Stackless 3.4: Use async and other new language features * Optimized debugger: Run faster, particularly in multi-process and multi-threaded code * Support for OS X full screen mode: Zoom to a virtual screen, with auto-hiding menu bar * Added a new One Dark color palette: Enjoy the best dark display style yet * Updated French and German localizations: Thanks to Jean Sanchez, Laurent Fasnacht, and Christoph Heitkam For a more detailed overview of new features see the release notice at http://wingware.com/news/2017-06-29 Annual License Option Wing 6 adds the option of purchasing a lower-cost expiring annual license for Wing Pro. An annual license includes access to all available Wing Pro versions while it is valid, and then ceases to function until it is renewed. Pricing for annual licenses is US$ 179/user for Commercial Use and US$ 69/user for Non-Commercial Use. Perpetual licenses for Wing Pro will continue to be available at the same pricing. The cost of extending Support+Upgrades subscriptions on Non-Commercial Use perpetual licenses for Wing Pro has also been dropped from US$ 89 to US$ 39 per user. For details, see https://wingware.com/store/ Wing Personal is Free Wing Personal is now free and no longer requires a license to run. It now also includes the Source Browser, PyLint, and OS Commands tools, and supports the scripting API and Perspectives. However, Wing Personal does not include Wing Pro's advanced editing, debugging, testing and code management features, such as remote development, refactoring, find uses, version control, unit testing, interactive debug probe, multi-process and child process debugging, move program counter, conditional breakpoints, debug watch, framework-specific support (for Jupyter, Django, and others), find symbol in project, and other features. Links Release notice: http://wingware.com/news/2017-06-29 Downloads and Free Trial: http://wingware.com/downloads Buy: http://wingware.com/store/purchase Upgrade: https://wingware.com/store/upgrade Questions? Don't hesitate to email us at support at wingware.com. Thanks, -- Stephan Deibel Wingware | Python IDE The Intelligent Development Environment for Python Programmers wingware.com From charles.wilt at gmail.com Fri Jun 30 12:14:46 2017 From: charles.wilt at gmail.com (Charles Wilt) Date: Fri, 30 Jun 2017 10:14:46 -0600 Subject: Fwd: ftplib sending data out of order In-Reply-To: References: Message-ID: Hello, First off, I'm not a python guy....but I use a set of python scripts created a few years ago by somebody else to transfer source between the SVN repo on my PC and an IBM i (aka AS/400) system. Recently, multiple developers, including me, have started having intermittent issues whereby the source gets sometimes gets scrambled during the send from PC to IBM i; and it seems to be happening more and more often. The python script hasn't been changed, and I've been running 2.7.12; though I've since tried upgrading to 2.7.13. I used wireshark to capture the FTP session and have confirmed that the data is leaving the PC in the incorrect order. (screenshot attached) originally our script had the following: connection.storlines('STOR ' + ibmi_file, open(source_file.fullpath, 'rb')) Following advice from a online midrange community, I tired some other options for the mode & buffer on the open. Right now, ?I'm using the following, which seesm to have reduced the occurrences of the issue... connection.storlines('STOR ' + ibmi_file, open(source_file.fullpath, 'Ur',0)) I tired it with buffer=1 (line) and was still seeing data being sent out of order. Honestly, I suspect some sort of Windows issue; as Windows is the only thing that's changed recently. But I'm running Win10 and another developer that's seeing it is running Win8. I'd appreciate any guidance you may have. Thank you, Charles Wilt From rosuav at gmail.com Fri Jun 30 12:27:57 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Jul 2017 02:27:57 +1000 Subject: Teaching the "range" function in Python 3 In-Reply-To: References: <59567638$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Jul 1, 2017 at 2:17 AM, Stefan Ram wrote: > However, to my defense, I must say that in this post my intend > was to demonstrate what is happening /behind the curtains/ when > the ?for? loop is running, so in this special case, it might be > appropriate to use a function that otherwise should only be > called by the Python interpreter. (But I don't actually know > whether the ?for? loop uses ?__iter__? or ?iter? or some other > means.) It basically uses iter(). https://docs.python.org/3/c-api/object.html#c.PyObject_GetIter """This is equivalent to the Python expression iter(o).""" ChrisA From Irv at furrypants.com Fri Jun 30 13:07:52 2017 From: Irv at furrypants.com (Irv Kalb) Date: Fri, 30 Jun 2017 10:07:52 -0700 Subject: Teaching the "range" function in Python 3 In-Reply-To: References: Message-ID: <00E0E690-87BD-4A99-9079-DBD2852CD93B@furrypants.com> > On Jun 29, 2017, at 2:21 PM, Chris Angelico wrote: > > On Fri, Jun 30, 2017 at 6:57 AM, Irv Kalb wrote: >> I am wondering if other teachers have run into this. Is this a real problem? If so, is there any other way of explaining the concept without getting into the underlying details of how a generator works? Do you think it would be helpful to use the words "sequence of numbers" rather than talking about a list here - or would that be even more confusing to students? >> > > The easiest way is to use the word "collection" for things you iterate > over (rather than the concrete term "list"). So you can loop through > (or iterate over) a collection of explicitly given numbers: > Thanks to everyone who responded to my question about teaching the range function. I particularly like the ideas of using the words "collection" and "sequence"- those seem to be very clear. In my curriculum, I first go through a in detail discussion of the need for, the syntax of and the usage of lists. Then introduce for loops and explain how to iterate through a list (and I do use a naming convention). At the very end of my discussion of lists and loops, I talk about the additional use of "range". I do not want to go into any details of the underlying implementation of range, I just wanted a way to explain how it could be used in a for statement. (I understand that xrange does the same thing in Python 2 as range in Python 3, but for the sake of simplicity, I do not teach xrange in my Python 2 class.) Thanks for all the comments about range not being a generator - I'll be sure to steer clear of that word in my introductory class. Not sure how this got into a discussion of function calls, etc. Thanks very much, Irv From nw at ethz.swi Fri Jun 30 15:32:00 2017 From: nw at ethz.swi (Nikolaus Wirth) Date: Fri, 30 Jun 2017 19:32:00 +0000 Subject: these 2 fucking clowns In-Reply-To: References: <2598d840-6c2b-4507-af36-658224d8717c@googlegroups.com> <8354d529-907d-4b69-80b3-a2ebcda15698@googlegroups.com> <2c793ed0-ba29-4ef0-a8fa-8de496265881@googlegroups.com> Message-ID: >> One strike and you're out. >> Mark Lawrence Can you formally prove that? From Trump at whitehouse.gov Fri Jun 30 15:57:00 2017 From: Trump at whitehouse.gov (The Donald) Date: Fri, 30 Jun 2017 19:57:00 +0000 Subject: Fwd: ftplib sending data out of order In-Reply-To: References: Message-ID: dude, you're totally welcome. just make sure that you vote the current wiki admins out of office. THEY'RE FIRED ! From Trump at whitehouse.gov Fri Jun 30 16:03:00 2017 From: Trump at whitehouse.gov (The Donald) Date: Fri, 30 Jun 2017 20:03:00 +0000 Subject: Fwd: ftplib sending data out of order In-Reply-To: References: Message-ID: Charles , please look up the main python wiki here: https://practical-scheme.net/wiliki/wiliki.cgi?python From rosuav at gmail.com Fri Jun 30 16:13:46 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Jul 2017 06:13:46 +1000 Subject: Fwd: ftplib sending data out of order In-Reply-To: References: Message-ID: On Sat, Jul 1, 2017 at 6:03 AM, The Donald wrote: > Charles , please look up the main python wiki here: > > https://REDACTED/wiliki/wiliki.cgi?python This person has been spamming this list and others, dodging bans, and generally behaving in a way that proves he is no member of the Python community, but just a troll and a spammer. I recommend ignoring him, reporting his posts, or whatever you find most appropriate. ChrisA From tjreedy at udel.edu Fri Jun 30 16:22:05 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 30 Jun 2017 16:22:05 -0400 Subject: Teaching the "range" function in Python 3 In-Reply-To: <00E0E690-87BD-4A99-9079-DBD2852CD93B@furrypants.com> References: <00E0E690-87BD-4A99-9079-DBD2852CD93B@furrypants.com> Message-ID: On 6/30/2017 1:07 PM, Irv Kalb wrote: > Thanks to everyone who responded to my question about teaching the range function. range is a class, not a function in the strict sense. Classes represent concepts. Instances of classes represent instances of the concept. Range represent the concept 'arithmetic sequence', a sequence of numbers with a constant difference. To form an arithmetic sequence, start with a number and keep adding a constant. Surely, schoolkids still have experience doing this, counting up or down by a number: 0,3,6,9,12,15...; 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0 'liftoff'; 99 bottles of beer. Counting down from 100 by a larger number is an old challenge. 100, 97, 94, 91, 88/ > I particularly like the ideas of using the words "collection" and "sequence"- those seem to be very clear. > > In my curriculum, I first go through a in detail discussion of the need for, the syntax of and the usage of lists. 'list' represents the more general concept 'mutable sequence of arbitrary information objects'. -- Terry Jan Reedy From allcasesolutions at gmail.com Fri Jun 30 16:26:44 2017 From: allcasesolutions at gmail.com (Case Solution & Analysis) Date: Fri, 30 Jun 2017 13:26:44 -0700 (PDT) Subject: Case Solution: CEO Activism (A) by Michael W. Toffel, Aaron Chatterji, Julia Kelley Message-ID: <44c4b6c5-d732-4eb9-9840-a3f73000bfdb@googlegroups.com> Case Solution and Analysis of CEO Activism (A) by Michael W. Toffel, Aaron Chatterji, Julia Kelley, send email to casesolutionscentre(at)gmail(dot)com Case Study ID: 9-617-001 Get Case Study Solution and Analysis of CEO Activism (A) in a FAIR PRICE!! Our e-mail address is CASESOLUTIONSCENTRE (AT) GMAIL (DOT) COM. Please replace (at) by @ and (dot) by . YOU MUST WRITE FOLLOWING WHILE PLACING YOUR ORDER: Complete Case Study Name Authors Case Study ID Publisher of Case Study Your Requirements / Case Questions Note: Do not reply to this post because we do not monitor posts. If you need any other Case Solutions please send me an email. We can help you to get it. From allcasesolutions at gmail.com Fri Jun 30 16:28:59 2017 From: allcasesolutions at gmail.com (Case Solution & Analysis) Date: Fri, 30 Jun 2017 13:28:59 -0700 (PDT) Subject: Case Solution: Floodgate On the Hunt for Thunder Lizards by Rory McDonald, Alix Burke, Emma Franking, Nicole Tempest Message-ID: Case Solution and Analysis of Floodgate: On the Hunt for Thunder Lizards by Rory McDonald, Alix Burke, Emma Franking, Nicole Tempest, send email to casesolutionscentre(at)gmail(dot)com Case Study ID: 9-617-044 Get Case Solution and Analysis of Floodgate: On the Hunt for Thunder Lizards in a FAIR PRICE!! Our e-mail address is CASESOLUTIONSCENTRE (AT) GMAIL (DOT) COM. Please replace (at) by @ and (dot) by . YOU MUST WRITE FOLLOWING WHILE PLACING YOUR ORDER: Complete Case Study Name Authors Case Study ID Publisher of Case Study Your Requirements / Case Questions Note: Do not reply to this post because we do not monitor posts. If you need any Case Solution please send me an email. We can help you to get it. From ozovozovozo202 at gmail.com Fri Jun 30 16:30:25 2017 From: ozovozovozo202 at gmail.com (Debiller 777) Date: Fri, 30 Jun 2017 13:30:25 -0700 (PDT) Subject: Python installer Message-ID: Please help me with Python. I many,MANY TIMES tried to install it, but always had problem....These installers were glistchy. What do I mean? I mean that when I installed Python I almost always got wrong packed folders. For example I open python's folder click on python.exe aaaaaand.....I just get error that there is no module name 'encodings' but it is just in wrong folder-Lib. When I am trying to organize it, still I get some errors. Yeah python.exe works but tkinter does not imports, plus pip cant launch because of the import errors. And I am really tired of this!!! How can I fully expierence Python if installer installs wrong? How? I am really tired of this. I even checked PC on viruses and malwares, but no this glitch HAPPENS. Pleas help to fix this problem and normally program on Python!!! From allcasesolutions at gmail.com Fri Jun 30 16:33:52 2017 From: allcasesolutions at gmail.com (Case Solution & Analysis) Date: Fri, 30 Jun 2017 13:33:52 -0700 (PDT) Subject: Case Solution: From Start-Up to Grown-Up Nation The Future of the Israeli Innovation Ecosystem (Abridged) by Elie Ofek, Margot Eiran Message-ID: Case Solution and Analysis of From Start-Up to Grown-Up Nation: The Future of the Israeli Innovation Ecosystem (Abridged) by Elie Ofek, Margot Eiran, send email to casesolutionscentre(at)gmail(dot)com Case Study ID: 9-517-103 Get Case Study Solution and Analysis of From Start-Up to Grown-Up Nation: The Future of the Israeli Innovation Ecosystem (Abridged) in a FAIR PRICE!! Our e-mail address is CASESOLUTIONSCENTRE (AT) GMAIL (DOT) COM. Please replace (at) by @ and (dot) by . YOU MUST WRITE FOLLOWING WHILE PLACING YOUR ORDER: Complete Case Study Name Authors Case Study ID Publisher of Case Study Your Requirements / Case Questions Note: Do not reply to this post because we do not reply posts here. If you need any Case Solution please send us an email. We can help you to get it. From allcasesolutions at gmail.com Fri Jun 30 16:36:53 2017 From: allcasesolutions at gmail.com (Case Solution & Analysis) Date: Fri, 30 Jun 2017 13:36:53 -0700 (PDT) Subject: Case Solution: Carmichael Roberts To Create a Private Equity Firm? by Steven Rogers, Kenneth Cooper Message-ID: <5a5512a8-35a9-43f0-b737-a7251465a64a@googlegroups.com> Case Solution and Analysis of Carmichael Roberts: To Create a Private Equity Firm? by Steven Rogers, Kenneth Cooper, send email to casesolutionscentre(at)gmail(dot)com Case Study ID: 9-317-079 Get Case Study Solution and Analysis of Carmichael Roberts: To Create a Private Equity Firm? in a FAIR PRICE!! Our e-mail address is CASESOLUTIONSCENTRE (AT) GMAIL (DOT) COM. Please replace (at) by @ and (dot) by . YOU MUST WRITE FOLLOWING WHILE PLACING YOUR ORDER: Complete Case Study Name Authors Case Study ID Publisher of Case Study Your Requirements / Case Questions Note: Do not reply to this post because we do not reply to posts here. If you need any Case Solution please send us an email. We can help you to get it. From allcasesolutions at gmail.com Fri Jun 30 16:39:44 2017 From: allcasesolutions at gmail.com (Case Solution & Analysis) Date: Fri, 30 Jun 2017 13:39:44 -0700 (PDT) Subject: Case Solution: Delivering the Goods at Shippo by Jeffrey J. Bussgang, Jeffrey Rayport, Olivia Hull Message-ID: <59b9b6a0-90b6-4242-b5e5-6ec687483ee8@googlegroups.com> Case Solution and Analysis of Delivering the Goods at Shippo by Jeffrey J. Bussgang, Jeffrey Rayport, Olivia Hull, send email to casesolutionscentre(at)gmail(dot)com Case Study ID: 9-817-065 Get Case Study Solution and Analysis of Delivering the Goods at Shippo in a FAIR PRICE!! Our e-mail address is CASESOLUTIONSCENTRE (AT) GMAIL (DOT) COM. Please replace (at) by @ and (dot) by . YOU MUST WRITE FOLLOWING WHILE PLACING YOUR ORDER: Complete Case Study Name Authors Case Study ID Publisher of Case Study Your Requirements / Case Questions Note: Do not reply to this post because we do not reply to posts here. If you need any Case Solution please send us an email. We can help you to get it. From eryksun at gmail.com Fri Jun 30 17:16:15 2017 From: eryksun at gmail.com (eryk sun) Date: Fri, 30 Jun 2017 21:16:15 +0000 Subject: Python installer In-Reply-To: References: Message-ID: On Fri, Jun 30, 2017 at 8:30 PM, Debiller 777 wrote: > I just get error that there is no module name 'encodings' First make sure that neither PYTHONHOME nor PYTHONPATH are defined in your environment. To check this type `set python` in a command prompt. Neither variable should be listed. From python at mrabarnett.plus.com Fri Jun 30 23:25:40 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 1 Jul 2017 04:25:40 +0100 Subject: Teaching the "range" function in Python 3 In-Reply-To: References: <00E0E690-87BD-4A99-9079-DBD2852CD93B@furrypants.com> Message-ID: <034e34ff-eb86-d687-40cc-7a459bab5e94@mrabarnett.plus.com> On 2017-07-01 03:12, Stefan Ram wrote: > Terry Reedy writes: >>range is a class, not a function in the strict sense. > > ?the built-in function range() returns an iterator of integers? > > The Python Language Reference, Release 3.6.0, 8.3 The for statement > Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> range I think it's a holdover from Python 2: Python 2.7.10 (default, May 23 2015, 09:44:00) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> range >>> xrange From rosuav at gmail.com Fri Jun 30 23:35:51 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Jul 2017 13:35:51 +1000 Subject: Teaching the "range" function in Python 3 In-Reply-To: <034e34ff-eb86-d687-40cc-7a459bab5e94@mrabarnett.plus.com> References: <00E0E690-87BD-4A99-9079-DBD2852CD93B@furrypants.com> <034e34ff-eb86-d687-40cc-7a459bab5e94@mrabarnett.plus.com> Message-ID: On Sat, Jul 1, 2017 at 1:25 PM, MRAB wrote: > On 2017-07-01 03:12, Stefan Ram wrote: >> >> Terry Reedy writes: >>> >>> range is a class, not a function in the strict sense. >> >> >> ?the built-in function range() returns an iterator of integers? >> >> The Python Language Reference, Release 3.6.0, 8.3 The for statement >> > Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit > (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> range > > > I think it's a holdover from Python 2: > > Python 2.7.10 (default, May 23 2015, 09:44:00) [MSC v.1500 64 bit (AMD64)] > on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> range > >>>> xrange > I think you're right. It's no longer correct. Let's fix it. https://github.com/python/cpython/pull/2523 ChrisA