From rustompmody at gmail.com Thu Feb 1 00:49:42 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 31 Jan 2018 21:49:42 -0800 (PST) Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] In-Reply-To: References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> <8b9a503a-1a70-b4ac-00d0-68021445f715@mrabarnett.plus.com> <5A71DD650200001B0001D70A@smtp1.astron.nl> Message-ID: On Wednesday, January 31, 2018 at 11:17:45 PM UTC+5:30, Adriaan Renting wrote: > I am Dutch and after googling the term, I can confirm that the "Dutch > Reach" is taught in driving school here. > I was taught this maneuvre when getting my licence 20 years ago. > And in the Netherlands, we largely solve this problem by just having > everyone on a bike. ;-) > (We do ride cars as well) Good to have an existence proof for civilization Topic remains wildly OT until somebody adds one more piece to the jigsaw: "?statistical evidence that the practice of 'dutch-reach' has significantly reduced the instances of dooring?" From porton at narod.ru Thu Feb 1 00:50:14 2018 From: porton at narod.ru (Victor Porton) Date: Thu, 01 Feb 2018 07:50:14 +0200 Subject: Handle SIGINT in C and Python (Posting On Python-List Prohibited) References: <7eaf9dd8-236d-43c1-815c-1757be126030@googlegroups.com> <02ad997d-1ccf-4789-8f75-4bd60accc7f7@googlegroups.com> <10d4b0b7-1de8-4874-b723-32d058cfc006@googlegroups.com> Message-ID: Lawrence D?Oliveiro wrote: > On Thursday, February 1, 2018 at 5:57:58 PM UTC+13, Victor Porton wrote: >> I meant to call poll() from C code, not Python code. > > Do you need to use C code at all? Python is quite capable of handling this > . I already concluded that I can use Popen.communicate() instead of my library. So the issue is closed. -- Victor Porton - http://portonvictor.org From eryksun at gmail.com Thu Feb 1 00:59:18 2018 From: eryksun at gmail.com (eryk sun) Date: Thu, 1 Feb 2018 05:59:18 +0000 Subject: Handle SIGINT in C and Python (Posting On Python-List Prohibited) In-Reply-To: References: <7eaf9dd8-236d-43c1-815c-1757be126030@googlegroups.com> <02ad997d-1ccf-4789-8f75-4bd60accc7f7@googlegroups.com> <10d4b0b7-1de8-4874-b723-32d058cfc006@googlegroups.com> Message-ID: On Thu, Feb 1, 2018 at 4:57 AM, Victor Porton wrote: > Lawrence D?Oliveiro wrote: > >> On Thursday, February 1, 2018 at 8:10:24 AM UTC+13, Victor Porton wrote: >>> Lawrence D?Oliveiro wrote: >>> >>>> The usual behaviour for POSIX is that the call is aborted with EINTR >>>> after you get the signal. >>> >>> That poll() is interrupted does not imply that Python will run its >>> pythonic signal handler at the point of interruption. That is a problem. >> >> * Python calls poll() >> * poll() aborted with EINTR >> * Python runs your signal handler >> >> Versus native C code: >> >> * your code calls poll() >> * poll() aborted with EINTR >> * your signal handler is run >> >> Where is there a fundamental difference? > > I meant to call poll() from C code, not Python code. In this case when > poll() is aborted with EINTR, the pythonic signal handler does not run. An extension module should call PyErr_CheckSignals [1] when interrupted by EINTR. For example, here's the loop used to call poll() in the standard-library select module: async_err = 0; do { Py_BEGIN_ALLOW_THREADS errno = 0; poll_result = poll(self->ufds, self->ufd_len, (int)ms); Py_END_ALLOW_THREADS if (errno != EINTR) break; /* poll() was interrupted by a signal */ if (PyErr_CheckSignals()) { async_err = 1; break; } if (timeout >= 0) { timeout = deadline - _PyTime_GetMonotonicClock(); if (timeout < 0) { poll_result = 0; break; } ms = _PyTime_AsMilliseconds(timeout, _PyTime_ROUND_CEILING); /* retry poll() with the recomputed timeout */ } } while (1); self->poll_running = 0; if (poll_result < 0) { if (!async_err) PyErr_SetFromErrno(PyExc_OSError); return NULL; } [1]: https://docs.python.org/3/c-api/exceptions.html#c.PyErr_CheckSignals From porton at narod.ru Thu Feb 1 01:01:07 2018 From: porton at narod.ru (Victor Porton) Date: Thu, 01 Feb 2018 08:01:07 +0200 Subject: Dependency injection: overriding defaults Message-ID: I am writing a library, a command line utility which uses the library, and a daemon which uses the library. I am going to use dependency_injector package. Consider loggers: For the core library the logger should default to stderr. For the command line utility, we use the default logger of the library. For the server, the log should go to a file (not to stderr). Question: How to profoundly make my software to use the appropriate logger, dependently on whether it is a command line utility or the daemon? -- Victor Porton - http://portonvictor.org From rustompmody at gmail.com Thu Feb 1 01:07:06 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 31 Jan 2018 22:07:06 -0800 (PST) Subject: Help to debug my free library In-Reply-To: References: <6fcc2db7-649f-4987-86df-6a73c21c91d7@googlegroups.com> Message-ID: On Thursday, February 1, 2018 at 1:11:50 AM UTC+5:30, Victor Porton wrote: > wxjmfauth wrote: > > > Le mercredi 31 janvier 2018 20:13:06 UTC+1, Chris Angelico a ?crit : > >> On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton wrote: > >> > LibComCom is a C library which passes a string as stdin of an OS > >> > command and stores its stdout in another string. > >> > >> Something like the built-in subprocess module does? > >> > >> ChrisA > > > > Do you mean the buggy subprocess module (coding of characters) ? > > Yes, there is a working workaround : QtCore.QProcess(). > > Please elaborate: which bugs it has? in which versions? > jmf is a pillar of the community who runs a campaign for equity and justice In particular that a $ costs just 0x24 (6 significant bits) whereas a ? costs 0x20AC (14 bits) and (3 bytes in UTF-8 n0xE2 0x82 0xAC) is highly unacceptable to him and he is taking it up with the UN. We must heartily support him in this noble endeavour [And when you get a chance to get your word in edgeways do ask him what this has to do with python] From porton at narod.ru Thu Feb 1 01:26:33 2018 From: porton at narod.ru (Victor Porton) Date: Thu, 01 Feb 2018 08:26:33 +0200 Subject: Object-oriented gettext Message-ID: I want to write a multiuser application which uses multiple languages (one language for logging and a language per user). https://docs.python.org/3/library/gettext.html describes a procedural gettext interface. The language needs to be switched before each gettext() call. I want an object oriented interface like: english.gettext("Word") == "Word" russian.gettext("Word") == "?????" That is, I do no want to write any language-switching code, but the language should depend on the object (like "english" and "russian" in the above example). What is the best way to do this? Should I write an object-oriented wrapper around gettext package? -- Victor Porton - http://portonvictor.org From porton at narod.ru Thu Feb 1 01:44:13 2018 From: porton at narod.ru (Victor Porton) Date: Thu, 01 Feb 2018 08:44:13 +0200 Subject: Advice on where to define dependency injection providers Message-ID: I define ExecutionContext in xmlboiler.core.execution_context module. ExecutionContext is meant to contain a common "environment" suitable for different kinds of tasks. Currently ExecutionContext contains a logger and a translator of messages: class ExecutionContext(object): def __init__(self, logger, translations): """ :param logger: logger :param translations: usually should be gettext.GNUTranslations """ self.logger = logger self.translations = translations Now I want to define some "provides" using dependency_injector.providers module. Where (in which module) should I define default factories for loggers, translations, and execution contexts? (Default logger should log to stderr, default translations should respect LANG environment variable.) The only quite clear thing is that providers should be defined somewhere in xmlboiler.core.* namespace, because it is the namespace for the core library (to be used by several applications). Should I define providers in xmlboiler.core.execution_context module or in something like xmlboiler.core.execution_context_build, or maybe in something like xmlboiler.core.providers.execution_context? -- Victor Porton - http://portonvictor.org From porton at narod.ru Thu Feb 1 02:06:25 2018 From: porton at narod.ru (Victor Porton) Date: Thu, 01 Feb 2018 09:06:25 +0200 Subject: Object-oriented gettext References: Message-ID: Victor Porton wrote: > I want to write a multiuser application which uses multiple languages (one > language for logging and a language per user). > > https://docs.python.org/3/library/gettext.html describes a procedural > gettext interface. The language needs to be switched before each gettext() > call. > > I want an object oriented interface like: > > english.gettext("Word") == "Word" > russian.gettext("Word") == "?????" > > That is, I do no want to write any language-switching code, but the > language should depend on the object (like "english" and "russian" in the > above example). > > What is the best way to do this? > > Should I write an object-oriented wrapper around gettext package? Oh, I see that gettext.translation() seems to do the job. -- Victor Porton - http://portonvictor.org From dieter at handshake.de Thu Feb 1 02:17:46 2018 From: dieter at handshake.de (dieter) Date: Thu, 01 Feb 2018 08:17:46 +0100 Subject: Dependency injection: overriding defaults References: Message-ID: <874ln1yy91.fsf@handshake.de> Victor Porton writes: > I am writing a library, a command line utility which uses the library, and a > I am going to use dependency_injector package. > > Consider loggers: > > For the core library the logger should default to stderr. > > For the command line utility, we use the default logger of the library. > > For the server, the log should go to a file (not to stderr). > > Question: How to profoundly make my software to use the appropriate logger, > dependently on whether it is a command line utility or the daemon? I would distinguish between the common library and distinct applications (command line utility, daemon). The applications configure the logging system (differently) while the library uses uniform logging calls. From naveen at emagevisionpl.com Thu Feb 1 03:50:28 2018 From: naveen at emagevisionpl.com (naveen at emagevisionpl.com) Date: Thu, 1 Feb 2018 00:50:28 -0800 (PST) Subject: auto-correct a speech-to-text output and relate to of the words based on syllables Message-ID: <78b57c55-b57f-4e80-b390-5fb56df4012b@googlegroups.com> Hi, I have to make an application in which, The user asks a question, Google's API is used to convert the speech to text. But the problem is due to different accent the translator misunderstands the words. I want my application to guess the word to the nearest word spoken by the user. When the user speaks a word, the syllables in the word should be matched with the syllables of other words in my dictionary. Till now what I have done, I used CMU pronouncing dictionary. It contains ~122300 words along with its syllables. Along with this, I have created my own dictionary which consists of words that I need only. I have implemented a python logic that makes reasonable guesses for the words user speaks and matches with my dictionary. But sometimes for more than a word it makes the same guess. example: The user speaks "Light". The system translates it as "Bright" The user speaks "White" The system translates it as "Bright" I don't want this to happen. This system should make a reasonable conversion. Any help would be appreciated. I am using python for logic. Regards, Naveen BM From tkadm30 at yandex.com Thu Feb 1 04:13:29 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Thu, 1 Feb 2018 04:13:29 -0500 Subject: PyPy support breaking CPython compatibility? In-Reply-To: <683c5cb5-f0b3-af46-7717-f806a4de900e@nedbatchelder.com> References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> <683c5cb5-f0b3-af46-7717-f806a4de900e@nedbatchelder.com> Message-ID: Le 2018-01-31 ? 05:21, Ned Batchelder a ?crit?: > On 1/30/18 3:58 PM, Etienne Robillard wrote: >> Hi Ned, >> >> >> Le 2018-01-30 ? 15:14, Ned Batchelder a ?crit?: >>> I'm curious what you had to change for PyPy? (Unless it's a Py2/Py3 >>> thing as Chris mentions.) >> >> Please take a look at the changesets: >> >> https://bitbucket.org/tkadm30/libschevo/commits/745d1aeab5c6ee0d336790cf13d16f327e10c2f8 >> >> https://bitbucket.org/tkadm30/libdurus/commits/875636e9b6caa840fd50ca87d69217d87fc06f43 >> >> >> In short, it seems PyPy automagically adds a __weakref__ attribute to >> __slots__, causing the CPython interpreter to raise a TypeError... > > PyPy and CPython are separate implementations, that never mix: you run > your code in one, or in the other.? How can PyPy do something at > runtime (add a __weakref__ attribute to __slots__) that can cause > CPython to do something at runtime (raise a TypeError)? Sorry Ned for the confusion. This is strictly a PyPy issue. CPython does not raise a TypeError. > > A small demonstration case would be very helpful for figuring out what > is going on. Yes.... The problem is with the following code on PyPy 5.9: class MyClass(object): ? __slots__ = ['__weakref__',] > > --Ned. -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From yahya-abou-imran at protonmail.com Thu Feb 1 07:38:14 2018 From: yahya-abou-imran at protonmail.com (Yahya Abou 'Imran) Date: Thu, 01 Feb 2018 07:38:14 -0500 Subject: Problem with coroutines old-style / new-style usage and features Message-ID: Hi guys. I am discovering coroutines and asynchronous programming, and I have a little problem with a little example I'm coding myself as an excercice. Let say you take two guys in the street: Dave and Bryan. You ask dave to count from 1 to 50, 1 by 1. He will do it fast. And you ask Bryan to count from 208 to 166 in reversing order, 7 by 7! It will take him some time between each number to think about it. Now I have a recorder wich is able to recognize voices. I use it to record both of them counting at the same time. Here is the recorder: @asyncio.coroutine def recorder(): dialog = [] while True: sent = yield dialog if sent is not None: name, things = sent dialog.append(f'{name} says : {things}') It is storing the dialog, and you can ask him fot it later by sending None to it. For the calculation, I'm using a ascyn generator: async def calcul_mental(range_args, name, timeout=0.2): for i in range(*range_args): await asyncio.sleep(timeout) yield name, i To link the two, I came up with this little coroutine: async def record(recorder, gen): async for name, i in gen: recorder.send([name, i]) And my main: def main(): g1 = calcul_mental([1, 51], name='Dave', timeout=0.2) g2 = calcul_mental([208, 165, -7], name='Bryan', timeout=2) r = recorder() r.send(None) coros = asyncio.gather(record(r, g1), record(r, g2)) loop = asyncio.get_event_loop() loop.run_until_complete(coros) dialog = r.send(None) for line in dialog: print(line) It works well, but I was wondering if I could turn recorder into a new style coroutine... The problems are: - I can't await for an async generator; - I can't let an await alone to send data to it; - I can't turn it into an AsyncGenerator because it will lost the .send() method. I think it's just a problem of design, but I wasn't able to solve it myself. Any thoughts about it? Thanks! From nimbiotics at gmail.com Thu Feb 1 09:04:51 2018 From: nimbiotics at gmail.com (Mario R. Osorio) Date: Thu, 1 Feb 2018 06:04:51 -0800 (PST) Subject: for info In-Reply-To: <38684a61-8145-48a5-b065-70d9aa43cdc2@googlegroups.com> References: <38684a61-8145-48a5-b065-70d9aa43cdc2@googlegroups.com> Message-ID: On Wednesday, January 31, 2018 at 10:55:59 AM UTC-5, M.Haroon Ali wrote: > from where we learn python for free of cost. i am begineer in python.plzz help me And after you're done with the OFFICIAL tutorials; there thousands of excellent free tutorials online. Just do some research. If you're more of the 'visual' person, you can also find tutorials in youtube. http://udemy.com also offers many free courses. HTH From porton at narod.ru Thu Feb 1 11:55:55 2018 From: porton at narod.ru (Victor Porton) Date: Thu, 01 Feb 2018 18:55:55 +0200 Subject: Dependency injection: overriding defaults References: <874ln1yy91.fsf@handshake.de> Message-ID: dieter wrote: > Victor Porton writes: > >> I am writing a library, a command line utility which uses the library, >> and a I am going to use dependency_injector package. >> >> Consider loggers: >> >> For the core library the logger should default to stderr. >> >> For the command line utility, we use the default logger of the library. >> >> For the server, the log should go to a file (not to stderr). >> >> Question: How to profoundly make my software to use the appropriate >> logger, dependently on whether it is a command line utility or the >> daemon? > > I would distinguish between the common library and distinct > applications (command line utility, daemon). The applications > configure the logging system (differently) while the library uses > uniform logging calls. You have essentially just repeated my requirements. But HOW to do this (using dependency_injector module)? -- Victor Porton - http://portonvictor.org From markb77 at gmail.com Thu Feb 1 12:01:35 2018 From: markb77 at gmail.com (superchromix) Date: Thu, 1 Feb 2018 09:01:35 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups Message-ID: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, started receiving spam messages several months ago. Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. When trying to access comp.lang.idl-pvwave, a message is now displayed, stating that the group owner needs to remove the spam, and can then apply to Google in order to have access reinstated. However, old public Usenet groups like this have no owner. The comp.lang.idl-pvwave group is more than 20 years old. Hence, there is no way to unblock the group. This is a serious problem, since the entire collection of postings going back many years has been blocked, no just the spam. This resource is frequently used by IDL programmers. Seeing the spam postings in this newsgroup, I expect something similar may happen to comp.lang.python, soon. From alister.ware at ntlworld.com Thu Feb 1 12:06:55 2018 From: alister.ware at ntlworld.com (alister) Date: Thu, 01 Feb 2018 17:06:55 GMT Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: On Thu, 01 Feb 2018 09:01:35 -0800, superchromix wrote: > Our own programming discussion newsgroup, located at > comp.lang.idl-pvwave, started receiving spam messages several months > ago. > > Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google > Groups. > > When trying to access comp.lang.idl-pvwave, a message is now displayed, > stating that the group owner needs to remove the spam, and can then > apply to Google in order to have access reinstated. > > However, old public Usenet groups like this have no owner. The > comp.lang.idl-pvwave group is more than 20 years old. Hence, there is > no way to unblock the group. > > This is a serious problem, since the entire collection of postings going > back many years has been blocked, no just the spam. This resource is > frequently used by IDL programmers. > > Seeing the spam postings in this newsgroup, I expect something similar > may happen to comp.lang.python, soon. simple solution stop using google groups & use either the mailing list or a news server with an NNTP client -- Your wig steers the gig. -- Lord Buckley From rosuav at gmail.com Thu Feb 1 12:15:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Feb 2018 04:15:54 +1100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: On Fri, Feb 2, 2018 at 4:01 AM, superchromix wrote: > > Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, started receiving spam messages several months ago. > > Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. > > When trying to access comp.lang.idl-pvwave, a message is now displayed, stating that the group owner needs to remove the spam, and can then apply to Google in order to have access reinstated. > > However, old public Usenet groups like this have no owner. The comp.lang.idl-pvwave group is more than 20 years old. Hence, there is no way to unblock the group. > > This is a serious problem, since the entire collection of postings going back many years has been blocked, no just the spam. This resource is frequently used by IDL programmers. > > Seeing the spam postings in this newsgroup, I expect something similar may happen to comp.lang.python, soon. > Pot, meet Kettle. Most of our worst spam comes FROM the Google Groups bridge. If they cut the connection, I would not be sorry. ChrisA From tdldev at gmail.com Thu Feb 1 12:21:54 2018 From: tdldev at gmail.com (Jack Dangler) Date: Thu, 1 Feb 2018 12:21:54 -0500 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: <43c86726-34a5-1004-1521-7fe38fccd0b8@gmail.com> On 02/01/2018 12:06 PM, alister via Python-list wrote: > On Thu, 01 Feb 2018 09:01:35 -0800, superchromix wrote: > >> Our own programming discussion newsgroup, located at >> comp.lang.idl-pvwave, started receiving spam messages several months >> ago. >> >> Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google >> Groups. >> >> When trying to access comp.lang.idl-pvwave, a message is now displayed, >> stating that the group owner needs to remove the spam, and can then >> apply to Google in order to have access reinstated. >> >> However, old public Usenet groups like this have no owner. The >> comp.lang.idl-pvwave group is more than 20 years old. Hence, there is >> no way to unblock the group. >> >> This is a serious problem, since the entire collection of postings going >> back many years has been blocked, no just the spam. This resource is >> frequently used by IDL programmers. >> >> Seeing the spam postings in this newsgroup, I expect something similar >> may happen to comp.lang.python, soon. > simple solution stop using google groups & use either the mailing list or > a news server with an NNTP client > > > Is it possible to name an 'owner' and then get into the group archive to remove the spam by account "recovery" ? From mail at timgolden.me.uk Thu Feb 1 12:25:04 2018 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 1 Feb 2018 17:25:04 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: On 01/02/2018 17:01, superchromix wrote: > > Our own programming discussion newsgroup, located at > comp.lang.idl-pvwave, started receiving spam messages several months > ago. > > Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google > Groups. > > When trying to access comp.lang.idl-pvwave, a message is now > displayed, stating that the group owner needs to remove the spam, and > can then apply to Google in order to have access reinstated. > > However, old public Usenet groups like this have no owner. The > comp.lang.idl-pvwave group is more than 20 years old. Hence, there > is no way to unblock the group. > > This is a serious problem, since the entire collection of postings > going back many years has been blocked, no just the spam. This > resource is frequently used by IDL programmers. > > Seeing the spam postings in this newsgroup, I expect something > similar may happen to comp.lang.python, soon. > Thanks for the heads-up. I'm not sure there's anything we'll be able to do. Google have seemed remarkably uninterested in Google Groups for some years now. Although I recognise how useful GG is for some people, I wouldn't be sorry [as a list maintainer here] if it were to be blocked for good by forces outside our control. We've wondered several times whether we ought to be block its posts outright, but to date we've simply used refined filters to block the spam and other abuse. If Google essentially close the group down, the decision is taken out of our hands. TJG From markb77 at gmail.com Thu Feb 1 12:50:26 2018 From: markb77 at gmail.com (markb77 at gmail.com) Date: Thu, 1 Feb 2018 09:50:26 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: <0d5d4eed-cec3-41b1-9676-262cc3a80f78@googlegroups.com> On Thursday, February 1, 2018 at 6:01:58 PM UTC+1, superchromix wrote: > Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, started receiving spam messages several months ago. > > Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. > > When trying to access comp.lang.idl-pvwave, a message is now displayed, stating that the group owner needs to remove the spam, and can then apply to Google in order to have access reinstated. > > However, old public Usenet groups like this have no owner. The comp.lang.idl-pvwave group is more than 20 years old. Hence, there is no way to unblock the group. > > This is a serious problem, since the entire collection of postings going back many years has been blocked, no just the spam. This resource is frequently used by IDL programmers. > > Seeing the spam postings in this newsgroup, I expect something similar may happen to comp.lang.python, soon. The problem I have now is that there is no public, searchable archive of comp.lang.idl-pvwave available. This was the real benefit of Google groups, from my point of view. There is something called "narkive", but its search function seems to be broken, and it doesn't archive very far back in time. From markb77 at gmail.com Thu Feb 1 12:52:14 2018 From: markb77 at gmail.com (markb77 at gmail.com) Date: Thu, 1 Feb 2018 09:52:14 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <43c86726-34a5-1004-1521-7fe38fccd0b8@gmail.com> Message-ID: <87d954d6-bed2-482e-864e-f642ae66008d@googlegroups.com> On Thursday, February 1, 2018 at 6:22:22 PM UTC+1, Jack Dangler wrote: > On 02/01/2018 12:06 PM, alister via Python-list wrote: > > On Thu, 01 Feb 2018 09:01:35 -0800, superchromix wrote: > > > >> Our own programming discussion newsgroup, located at > >> comp.lang.idl-pvwave, started receiving spam messages several months > >> ago. > >> > >> Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google > >> Groups. > >> > >> When trying to access comp.lang.idl-pvwave, a message is now displayed, > >> stating that the group owner needs to remove the spam, and can then > >> apply to Google in order to have access reinstated. > >> > >> However, old public Usenet groups like this have no owner. The > >> comp.lang.idl-pvwave group is more than 20 years old. Hence, there is > >> no way to unblock the group. > >> > >> This is a serious problem, since the entire collection of postings going > >> back many years has been blocked, no just the spam. This resource is > >> frequently used by IDL programmers. > >> > >> Seeing the spam postings in this newsgroup, I expect something similar > >> may happen to comp.lang.python, soon. > > simple solution stop using google groups & use either the mailing list or > > a news server with an NNTP client > > > > > > > Is it possible to name an 'owner' and then get into the group archive to > remove the spam by account "recovery" ? Is this possible within the framework of the Usenet? From rosuav at gmail.com Thu Feb 1 13:01:21 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Feb 2018 05:01:21 +1100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: <0d5d4eed-cec3-41b1-9676-262cc3a80f78@googlegroups.com> References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <0d5d4eed-cec3-41b1-9676-262cc3a80f78@googlegroups.com> Message-ID: On Fri, Feb 2, 2018 at 4:50 AM, wrote: > On Thursday, February 1, 2018 at 6:01:58 PM UTC+1, superchromix wrote: >> Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, started receiving spam messages several months ago. >> >> Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. >> >> When trying to access comp.lang.idl-pvwave, a message is now displayed, stating that the group owner needs to remove the spam, and can then apply to Google in order to have access reinstated. >> >> However, old public Usenet groups like this have no owner. The comp.lang.idl-pvwave group is more than 20 years old. Hence, there is no way to unblock the group. >> >> This is a serious problem, since the entire collection of postings going back many years has been blocked, no just the spam. This resource is frequently used by IDL programmers. >> >> Seeing the spam postings in this newsgroup, I expect something similar may happen to comp.lang.python, soon. > > The problem I have now is that there is no public, searchable archive of comp.lang.idl-pvwave available. This was the real benefit of Google groups, from my point of view. > > There is something called "narkive", but its search function seems to be broken, and it doesn't archive very far back in time. > Which is why you don't rely on proprietary services if you don't have to. They can go away at any time, without notice. Fortunately, python-list has its own archive, so you can access that from python.org independently of the newsgroup's availability elsewhere on the internet. It doesn't have all the posts that are on the newsgroup and not on the mailing list, but IMO that's a feature, not a bug. (With the possible exception of the "no-archive" posts. Possible exception only.) ChrisA From none at invalid.com Thu Feb 1 13:05:31 2018 From: none at invalid.com (mm0fmf) Date: Thu, 01 Feb 2018 18:05:31 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: On 01/02/2018 17:15, Chris Angelico wrote: > On Fri, Feb 2, 2018 at 4:01 AM, superchromix wrote: >> >> Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, started receiving spam messages several months ago. >> >> Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. >> >> When trying to access comp.lang.idl-pvwave, a message is now displayed, stating that the group owner needs to remove the spam, and can then apply to Google in order to have access reinstated. >> >> However, old public Usenet groups like this have no owner. The comp.lang.idl-pvwave group is more than 20 years old. Hence, there is no way to unblock the group. >> >> This is a serious problem, since the entire collection of postings going back many years has been blocked, no just the spam. This resource is frequently used by IDL programmers. >> >> Seeing the spam postings in this newsgroup, I expect something similar may happen to comp.lang.python, soon. >> > > Pot, meet Kettle. > > Most of our worst spam comes FROM the Google Groups bridge. If they > cut the connection, I would not be sorry. > ^^^^^^^^^^^^^^^^^^^^^^ This. This a googol times! From sean.dizazzo at gmail.com Thu Feb 1 13:22:41 2018 From: sean.dizazzo at gmail.com (Sean DiZazzo) Date: Thu, 1 Feb 2018 10:22:41 -0800 (PST) Subject: Create an alias to an attribute on superclass Message-ID: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> Hi! I basically just want to create an alias to an attribute on an item's superclass. So that after I create the subclass object, I can access the alias attribute to get the value back. 
class Superclass(object):
    def __init__(self, value):
        """
            I want to pass x by reference, so that any time
            x on the subclass is updated, so is the alias here
        """
        self.alias = value

class Subclass(Superclass):
    def __init__(self, x):
        self.x = x
        Superclass.__init__(self, self.x)

    def __repr__(self):
        return "x: %s\nalias: %s" % (self.x, self.alias)


if __name__ == "__main__":
    foo = Subclass(1)
    print foo.alias

    foo.x = 6
    # Should return 6 !!!
    print foo.alias
From rosuav at gmail.com Thu Feb 1 13:36:58 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Feb 2018 05:36:58 +1100 Subject: Create an alias to an attribute on superclass In-Reply-To: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> References: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> Message-ID: On Fri, Feb 2, 2018 at 5:22 AM, Sean DiZazzo wrote: > Hi! > > I basically just want to create an alias to an attribute on an item's superclass. So that after I create the subclass object, I can access the alias attribute to get the value back. > > 
> class Superclass(object):
>     def __init__(self, value):
>         """
>             I want to pass x by reference, so that any time
>             x on the subclass is updated, so is the alias here
>         """
>         self.alias = value
>
> class Subclass(Superclass):
>     def __init__(self, x):
>         self.x = x
>         Superclass.__init__(self, self.x)
>
>     def __repr__(self):
>         return "x: %s\nalias: %s" % (self.x, self.alias)
>
>
> if __name__ == "__main__":
>     foo = Subclass(1)
>     print foo.alias
>
>     foo.x = 6
>     # Should return 6 !!!
>     print foo.alias
>
>
ISTM the easiest way would be to define a property on the superclass: class Superclass(object): @property def alias(self): return self.x Whatever happens, self.alias will be identical to self.x. Is that what you're after? ChrisA From sean.dizazzo at gmail.com Thu Feb 1 13:44:20 2018 From: sean.dizazzo at gmail.com (Sean DiZazzo) Date: Thu, 1 Feb 2018 10:44:20 -0800 (PST) Subject: Create an alias to an attribute on superclass In-Reply-To: References: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> Message-ID: <520ea7bb-654f-480c-8e94-3f6a56350440@googlegroups.com> On Thursday, February 1, 2018 at 10:37:32 AM UTC-8, Chris Angelico wrote: > On Fri, Feb 2, 2018 at 5:22 AM, Sean DiZazzo wrote: > > Hi! > > > > I basically just want to create an alias to an attribute on an item's superclass. So that after I create the subclass object, I can access the alias attribute to get the value back. > > > > 
> > class Superclass(object):
> >     def __init__(self, value):
> >         """
> >             I want to pass x by reference, so that any time
> >             x on the subclass is updated, so is the alias here
> >         """
> >         self.alias = value
> >
> > class Subclass(Superclass):
> >     def __init__(self, x):
> >         self.x = x
> >         Superclass.__init__(self, self.x)
> >
> >     def __repr__(self):
> >         return "x: %s\nalias: %s" % (self.x, self.alias)
> >
> >
> > if __name__ == "__main__":
> >     foo = Subclass(1)
> >     print foo.alias
> >
> >     foo.x = 6
> >     # Should return 6 !!!
> >     print foo.alias
> >
> >
> > ISTM the easiest way would be to define a property on the superclass: > > class Superclass(object): > @property > def alias(self): > return self.x > > Whatever happens, self.alias will be identical to self.x. Is that what > you're after? > > ChrisA Yes, but that doesn't seem to work. It looks like there is a way to do it if you make the original value a list (mutable) instead of an integer. I really need to do it with an arbitrary object. Not sure if I can hack the list trick to work in my case though. From rosuav at gmail.com Thu Feb 1 13:49:44 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Feb 2018 05:49:44 +1100 Subject: Create an alias to an attribute on superclass In-Reply-To: <520ea7bb-654f-480c-8e94-3f6a56350440@googlegroups.com> References: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> <520ea7bb-654f-480c-8e94-3f6a56350440@googlegroups.com> Message-ID: On Fri, Feb 2, 2018 at 5:44 AM, Sean DiZazzo wrote: > On Thursday, February 1, 2018 at 10:37:32 AM UTC-8, Chris Angelico wrote: >> On Fri, Feb 2, 2018 at 5:22 AM, Sean DiZazzo wrote: >> > Hi! >> > >> > I basically just want to create an alias to an attribute on an item's superclass. So that after I create the subclass object, I can access the alias attribute to get the value back. >> > >> > 
>> > class Superclass(object):
>> >     def __init__(self, value):
>> >         """
>> >             I want to pass x by reference, so that any time
>> >             x on the subclass is updated, so is the alias here
>> >         """
>> >         self.alias = value
>> >
>> > class Subclass(Superclass):
>> >     def __init__(self, x):
>> >         self.x = x
>> >         Superclass.__init__(self, self.x)
>> >
>> >     def __repr__(self):
>> >         return "x: %s\nalias: %s" % (self.x, self.alias)
>> >
>> >
>> > if __name__ == "__main__":
>> >     foo = Subclass(1)
>> >     print foo.alias
>> >
>> >     foo.x = 6
>> >     # Should return 6 !!!
>> >     print foo.alias
>> >
>> >
>> >> ISTM the easiest way would be to define a property on the superclass: >> >> class Superclass(object): >> @property >> def alias(self): >> return self.x >> >> Whatever happens, self.alias will be identical to self.x. Is that what >> you're after? >> >> ChrisA > > Yes, but that doesn't seem to work. It looks like there is a way to do it if you make the original value a list (mutable) instead of an integer. I really need to do it with an arbitrary object. Not sure if I can hack the list trick to work in my case though. > Can you post your actual code? The alias will be looked up whenever you ask for it, so it won't be a copy. ChrisA From ian.g.kelly at gmail.com Thu Feb 1 13:50:58 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 1 Feb 2018 11:50:58 -0700 Subject: Create an alias to an attribute on superclass In-Reply-To: <520ea7bb-654f-480c-8e94-3f6a56350440@googlegroups.com> References: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> <520ea7bb-654f-480c-8e94-3f6a56350440@googlegroups.com> Message-ID: On Thu, Feb 1, 2018 at 11:44 AM, Sean DiZazzo wrote: > On Thursday, February 1, 2018 at 10:37:32 AM UTC-8, Chris Angelico wrote: >> On Fri, Feb 2, 2018 at 5:22 AM, Sean DiZazzo wrote: >> > Hi! >> > >> > I basically just want to create an alias to an attribute on an item's superclass. So that after I create the subclass object, I can access the alias attribute to get the value back. >> > >> > 
>> > class Superclass(object):
>> >     def __init__(self, value):
>> >         """
>> >             I want to pass x by reference, so that any time
>> >             x on the subclass is updated, so is the alias here
>> >         """
>> >         self.alias = value
>> >
>> > class Subclass(Superclass):
>> >     def __init__(self, x):
>> >         self.x = x
>> >         Superclass.__init__(self, self.x)
>> >
>> >     def __repr__(self):
>> >         return "x: %s\nalias: %s" % (self.x, self.alias)
>> >
>> >
>> > if __name__ == "__main__":
>> >     foo = Subclass(1)
>> >     print foo.alias
>> >
>> >     foo.x = 6
>> >     # Should return 6 !!!
>> >     print foo.alias
>> >
>> >
>> >> ISTM the easiest way would be to define a property on the superclass: >> >> class Superclass(object): >> @property >> def alias(self): >> return self.x >> >> Whatever happens, self.alias will be identical to self.x. Is that what >> you're after? >> >> ChrisA > > Yes, but that doesn't seem to work. It looks like there is a way to do it if you make the original value a list (mutable) instead of an integer. I really need to do it with an arbitrary object. Not sure if I can hack the list trick to work in my case though. What about it doesn't work? Did you want to be able to set the value via the alias as well? Then use a property with a setter: class Superclass(object): @property def alias(self): return self.x @alias.setter def alias(self, value): self.x = value From sean.dizazzo at gmail.com Thu Feb 1 13:56:50 2018 From: sean.dizazzo at gmail.com (Sean DiZazzo) Date: Thu, 1 Feb 2018 10:56:50 -0800 (PST) Subject: Create an alias to an attribute on superclass In-Reply-To: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> References: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> Message-ID: <91927efa-1688-4261-840e-9f0f9b412404@googlegroups.com> On Thursday, February 1, 2018 at 10:23:06 AM UTC-8, Sean DiZazzo wrote: > Hi! > > I basically just want to create an alias to an attribute on an item's superclass. So that after I create the subclass object, I can access the alias attribute to get the value back. > > 
> class Superclass(object):
>     def __init__(self, value):
>         """
>             I want to pass x by reference, so that any time
>             x on the subclass is updated, so is the alias here
>         """
>         self.alias = value
> 
> class Subclass(Superclass):
>     def __init__(self, x):
>         self.x = x
>         Superclass.__init__(self, self.x)
> 
>     def __repr__(self):
>         return "x: %s\nalias: %s" % (self.x, self.alias)
> 
> 
> if __name__ == "__main__":
>     foo = Subclass(1)
>     print foo.alias
> 
>     foo.x = 6
>     # Should return 6 !!!
>     print foo.alias
> 
>
Yep. Thats it. Thank you guys!! From ben+python at benfinney.id.au Thu Feb 1 14:00:38 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 02 Feb 2018 06:00:38 +1100 Subject: Create an alias to an attribute on superclass References: <8dfe6f97-564d-49d8-871d-388b9e7c6177@googlegroups.com> Message-ID: <85wozwh6w9.fsf@benfinney.id.au> Sean DiZazzo writes: > I basically just want to create an alias to an attribute on an item's > superclass. This description ? ?attribute on an item's superclass? ? does not match what you have described in the rest of the message. Note the difference between an attribute ?spam? on a class ?Lorem? (accessed via ?Lorem.foo?, shared among all instances of ?Lorem?), versus an attribute ?spam? on a specific instance of ?Lorem?. For more information, read about how Python resolves an attribute reference from an instance, then to its class, and so on: A class instance has a namespace implemented as a dictionary which is the first place in which attribute references are searched. When an attribute is not found there, and the instance?s class has an attribute by that name, the search continues with the class attributes. So, if the instance has an attribute ?spam?, then a reference to that name will first find the instance's attribute. If not, the class attribute will be tried; and then that class's superclass; and so on through the inheritance tree. Typically, you bind an attribute on the class when you want that reference shared (e.g. a default) for all the instances of that class; and you bind an attribute on the instance when you want that reference to be specific to that instance. > So that after I create the subclass object, I can access the alias > attribute to get the value back. Any instance method can interrogate ?self.spam? to ask for that instance's ?spam? attribute. This is true regardless of which class defines that method. What problem are you experiencing, and why do you think the default attribute access behaviour is not enough? -- \ ?Sane people have an appropriate perspective on the relative | `\ importance of foodstuffs and human beings. Crazy people can't | _o__) tell the difference.? ?Paul Z. Myers, 2010-04-18 | Ben Finney From balglaas at dds.nl Thu Feb 1 14:10:00 2018 From: balglaas at dds.nl (Jan van den Broek) Date: Thu, 1 Feb 2018 19:10:00 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: On 2018-02-01, superchromix wrote: > > Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, >started receiving spam messages several months ago. > > Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. > > When trying to access comp.lang.idl-pvwave, a message is now displayed, >stating that the group owner needs to remove the spam, and can then apply >to Google in order to have access reinstated. [Schnipp] Personally I would see this as an improvement. -- Jan v/d Broek balglaas at dds.nl From ian.g.kelly at gmail.com Thu Feb 1 14:19:59 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 1 Feb 2018 12:19:59 -0700 Subject: Problem with coroutines old-style / new-style usage and features In-Reply-To: References: Message-ID: On Thu, Feb 1, 2018 at 5:38 AM, Yahya Abou 'Imran via Python-list wrote: > Hi guys. > > I am discovering coroutines and asynchronous programming, and I have a little problem with a little example I'm coding myself as an excercice. > > Let say you take two guys in the street: Dave and Bryan. > You ask dave to count from 1 to 50, 1 by 1. He will do it fast. > And you ask Bryan to count from 208 to 166 in reversing order, 7 by 7! It will take him some time between each number to think about it. > > Now I have a recorder wich is able to recognize voices. I use it to record both of them counting at the same time. > > Here is the recorder: > > > @asyncio.coroutine > def recorder(): > dialog = [] > while True: > sent = yield dialog > if sent is not None: > name, things = sent > dialog.append(f'{name} says : {things}') This is not an asyncio coroutine. This is just a normal generator that you're sending to. So you should probably remove the decorator to prevent confusion. For that matter I'm not really sure why this isn't just a class with synchronous "add" and "get" methods. > It is storing the dialog, and you can ask him fot it later by sending None to it. > > For the calculation, I'm using a ascyn generator: > > > async def calcul_mental(range_args, name, timeout=0.2): > for i in range(*range_args): > await asyncio.sleep(timeout) > yield name, i > > > To link the two, I came up with this little coroutine: > > > async def record(recorder, gen): > async for name, i in gen: > recorder.send([name, i]) > > > And my main: > > > def main(): > > g1 = calcul_mental([1, 51], > name='Dave', > timeout=0.2) > > g2 = calcul_mental([208, 165, -7], > name='Bryan', > timeout=2) > > r = recorder() > r.send(None) > > coros = asyncio.gather(record(r, g1), record(r, g2)) > loop = asyncio.get_event_loop() > loop.run_until_complete(coros) > > dialog = r.send(None) > for line in dialog: > print(line) > > > It works well, but I was wondering if I could turn recorder into a new style coroutine... Why? It doesn't do anything asynchronously. > The problems are: > - I can't await for an async generator; > - I can't let an await alone to send data to it; > - I can't turn it into an AsyncGenerator because it will lost the .send() method. > > I think it's just a problem of design, but I wasn't able to solve it myself. > > Any thoughts about it? If it were a class, then you could make the individual methods be coroutines if desired and await those. From grant.b.edwards at gmail.com Thu Feb 1 14:34:32 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 1 Feb 2018 19:34:32 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: On 2018-02-01, Jan van den Broek wrote: > On 2018-02-01, superchromix wrote: >> >> Our own programming discussion newsgroup, located at >> comp.lang.idl-pvwave, started receiving spam messages several >> months ago. >> >> Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google >> Groups. [...] > [Schnipp] > > Personally I would see this as an improvement. I plonked all posts from google groops yonks ago. Now that gmane's web UI is gone, maybe google groups is useful as a searchable archive (I don't know, I've never used it). For me, google groups is nothing but a source of spam. -- Grant Edwards grant.b.edwards Yow! Let's send the at Russians defective gmail.com lifestyle accessories! From yahya-abou-imran at protonmail.com Thu Feb 1 15:19:46 2018 From: yahya-abou-imran at protonmail.com (Yahya Abou 'Imran) Date: Thu, 01 Feb 2018 15:19:46 -0500 Subject: Problem with coroutines old-style / new-style usage and features In-Reply-To: References: Message-ID: >> @asyncio.coroutine >> def recorder(): >> dialog = [] >> while True: >> sent = yield dialog >> if sent is not None: >> name, things = sent >> dialog.append(f'{name} says : {things}') >This is not an asyncio coroutine. This is just a normal generator that > you're sending to. So you should probably remove the decorator to > prevent confusion. For that matter I'm not really sure why this isn't > just a class with synchronous "add" and "get" methods. Yeah it's true. I would just instantiate it, send to it and ask for a result. To pass it to next to start it and to retrieve a value is not really explicit. [...] >>It works well, but I was wondering if I could turn recorder into a new style coroutine... >Why? It doesn't do anything asynchronously. Hey, you're right... I was just, you know... "Async is so cool, let's just async everything!" >If it were a class, then you could make the individual methods be > coroutines if desired and await those. Thanks for your advise Ian! From william.sewell at wgu.edu Thu Feb 1 15:26:20 2018 From: william.sewell at wgu.edu (William Sewell) Date: Thu, 1 Feb 2018 20:26:20 +0000 Subject: Trace back error just trying to run a simple import of requests Message-ID: My python script which I run daily just blew up. So, I went into python to diagnose and just typed in the first line - import requests. I received screenfulls of errors. Why? What could have happened when nothing changed on my end? The traceback is copied below. [cid:image001.png at 01D39B71.014E8850] Dr. William Sewell Direct Line: 385.428.5377 Toll Free: 877.435.7948 ext 5377 Course Instructor, IT WGU will be closed President's Day, February 19th. Mountain Time Office hours: Sun 3:00 PM - 8:00 PM, Mon 6:30 AM - 3:30 PM, Tue 9:30 AM - 6:30 PM, Wed 9:00 AM - 5:00 PM, Thu 6:30 AM - 3:30 PM Your success is very important to me. IT MSDA and Database Team cmdatabase at wgu.edu Toll Free: 1-877-435-7948 Student Services: 1-866-903-0110 WellConnect Counseling Services: 1-877-685-3269 Western Governors University mailto:william.sewell at wgu.edu [WGU] More about WGU in Fast Company, CNN, NPR, NBC Nightly News, Money, The Atlantic, TIME, etc. wgu.edu From kushal at locationd.net Thu Feb 1 15:28:15 2018 From: kushal at locationd.net (Kushal Kumaran) Date: Thu, 01 Feb 2018 12:28:15 -0800 Subject: Dependency injection: overriding defaults In-Reply-To: (Victor Porton's message of "Thu, 01 Feb 2018 18:55:55 +0200") References: <874ln1yy91.fsf@handshake.de> Message-ID: <878tccjvz4.fsf@niobium.community.veritas.com> Victor Porton writes: > dieter wrote: > >> Victor Porton writes: >> >>> I am writing a library, a command line utility which uses the library, >>> and a I am going to use dependency_injector package. >>> >>> Consider loggers: >>> >>> For the core library the logger should default to stderr. >>> >>> For the command line utility, we use the default logger of the library. >>> >>> For the server, the log should go to a file (not to stderr). >>> >>> Question: How to profoundly make my software to use the appropriate >>> logger, dependently on whether it is a command line utility or the >>> daemon? >> >> I would distinguish between the common library and distinct >> applications (command line utility, daemon). The applications >> configure the logging system (differently) while the library uses >> uniform logging calls. > > You have essentially just repeated my requirements. > > But HOW to do this (using dependency_injector module)? > No idea how dependency_injector affects this, but have you read the logging howto? The section https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library would be appropriate. -- regards, kushal From petef4 at gmail.com Thu Feb 1 15:34:06 2018 From: petef4 at gmail.com (Pete Forman) Date: Thu, 01 Feb 2018 20:34:06 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <0d5d4eed-cec3-41b1-9676-262cc3a80f78@googlegroups.com> Message-ID: <87372k4fgh.fsf@gmail.com> markb77 at gmail.com writes: > On Thursday, February 1, 2018 at 6:01:58 PM UTC+1, superchromix wrote: >> Our own programming discussion newsgroup, located at >> comp.lang.idl-pvwave, started receiving spam messages several months >> ago. >> >> Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google >> Groups. >> >> When trying to access comp.lang.idl-pvwave, a message is now >> displayed, stating that the group owner needs to remove the spam, >> and can then apply to Google in order to have access reinstated. >> >> However, old public Usenet groups like this have no owner. The >> comp.lang.idl-pvwave group is more than 20 years old. Hence, there >> is no way to unblock the group. >> >> This is a serious problem, since the entire collection of postings >> going back many years has been blocked, no just the spam. This >> resource is frequently used by IDL programmers. >> >> Seeing the spam postings in this newsgroup, I expect something >> similar may happen to comp.lang.python, soon. > > The problem I have now is that there is no public, searchable archive > of comp.lang.idl-pvwave available. This was the real benefit of Google > groups, from my point of view. > > There is something called "narkive", but its search function seems to > be broken, and it doesn't archive very far back in time. A couple of other mail archivers are: https://www.mail-archive.com https://marc.info -- Pete Forman From drsalists at gmail.com Thu Feb 1 17:17:04 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 1 Feb 2018 14:17:04 -0800 Subject: for info In-Reply-To: <38684a61-8145-48a5-b065-70d9aa43cdc2@googlegroups.com> References: <38684a61-8145-48a5-b065-70d9aa43cdc2@googlegroups.com> Message-ID: On Wed, Jan 31, 2018 at 7:55 AM, wrote: > from where we learn python for free of cost. i am begineer in python.plzz help me If you already know another programming language: https://wiki.python.org/moin/BeginnersGuide/Programmers If you're making Python your first language: https://wiki.python.org/moin/BeginnersGuide/NonProgrammers HTH From grant.b.edwards at gmail.com Thu Feb 1 17:32:40 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 1 Feb 2018 22:32:40 +0000 (UTC) Subject: Trace back error just trying to run a simple import of requests References: Message-ID: On 2018-02-01, William Sewell wrote: > My python script which I run daily just blew up. So, I went into > python to diagnose and just typed in the first line - import > requests. I received screenfulls of errors. Why? What could have > happened when nothing changed on my end? The traceback is copied > below. > > [cid:image001.png at 01D39B71.014E8850] I don't recoginze "cid:image001.png at 01D39B71.014E8850" as an error message. Wow. This is the most bloated sig I've seen in decades... > Dr. William Sewell > Direct Line: 385.428.5377 Toll Free: 877.435.7948 ext 5377 > Course Instructor, IT > > WGU will be closed President's Day, February 19th. > Mountain Time Office hours: Sun 3:00 PM - 8:00 PM, Mon 6:30 AM - 3:30 PM, Tue 9:30 AM - 6:30 PM, Wed 9:00 AM - 5:00 PM, Thu 6:30 AM - 3:30 PM > Your success is very important to me. > IT MSDA and Database Team > cmdatabase at wgu.edu > Toll Free: 1-877-435-7948 > Student Services: 1-866-903-0110 > WellConnect Counseling Services: 1-877-685-3269 > Western Governors University > mailto:william.sewell at wgu.edu > > [WGU] > > More about WGU in Fast Company, CNN, NPR, NBC Nightly News, Money, The Atlantic, TIME, etc. > wgu.edu -- Grant Edwards grant.b.edwards Yow! Used staples are good at with SOY SAUCE! gmail.com From steve+comp.lang.python at pearwood.info Thu Feb 1 18:41:02 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 1 Feb 2018 23:41:02 +0000 (UTC) Subject: Trace back error just trying to run a simple import of requests References: Message-ID: On Thu, 01 Feb 2018 20:26:20 +0000, William Sewell wrote: > My python script which I run daily just blew up. So, I went into python > to diagnose and just typed in the first line - import requests. I > received screenfulls of errors. Why? What could have happened when > nothing changed on my end? Ah, the infamous "nothing changed, now things have changed!" error. Computers are deterministic, so something has changed. We may be able to help you find out what. > The traceback is copied below. If by chance you attached it as a screen shot, please don't: - screen shots of text are discriminatory against programmers who are visually impaired or blind; - they're also counter-productive, as we don't edit our code with photoshop; being able to edit copied text is important; and most importantly: - this mailing list is *text only* and blocks all non-text attachments (which are nearly always spam and malware), so it can't get through to us. Please copy and paste the text of the traceback and include it in the body of your reply (to the list, not to me personally). If for some reason you genuinely cannot copy the text, which I doubt as even Windows supports copy and paste, and you don't mind discriminating against the visually impaired, then you can upload the screen shot to an image hosting site (one without intrusive ads and malware!) and post the URL here. The only one I would use is imgur. Good luck. -- Steve From jladasky at itu.edu Thu Feb 1 18:48:36 2018 From: jladasky at itu.edu (jladasky at itu.edu) Date: Thu, 1 Feb 2018 15:48:36 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: <015584fe-197b-4958-a623-64106c17ae2c@googlegroups.com> On Thursday, February 1, 2018 at 9:07:15 AM UTC-8, alister wrote: > simple solution stop using google groups & use either the mailing list or > a news server with an NNTP client Sigh. I've been on Usenet since 1986. Is this how it ends? I will move if I must. Let me ask those of you who are not using Google Groups: how do you search? In my experience, searching through mailing list archives has been poor. From python at mrabarnett.plus.com Thu Feb 1 18:49:54 2018 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 1 Feb 2018 23:49:54 +0000 Subject: Trace back error just trying to run a simple import of requests In-Reply-To: References: Message-ID: <3a9fdd65-b4fb-bbdb-19e1-8a14a8199e1a@mrabarnett.plus.com> On 2018-02-01 22:32, Grant Edwards wrote: > On 2018-02-01, William Sewell wrote: > >> My python script which I run daily just blew up. So, I went into >> python to diagnose and just typed in the first line - import >> requests. I received screenfulls of errors. Why? What could have >> happened when nothing changed on my end? The traceback is copied >> below. >> >> [cid:image001.png at 01D39B71.014E8850] > > I don't recoginze "cid:image001.png at 01D39B71.014E8850" as an error > message. > Isn't it ironic that we've just had a thread about this very issue, namely sending screenshots? [snip] From steve+comp.lang.python at pearwood.info Thu Feb 1 18:58:26 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 1 Feb 2018 23:58:26 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <015584fe-197b-4958-a623-64106c17ae2c@googlegroups.com> Message-ID: On Thu, 01 Feb 2018 15:48:36 -0800, jladasky wrote: > Let me ask those of you who are not using Google Groups: how do you > search? In my experience, searching through mailing list archives has > been poor. https://duckduckgo.com/?q=site%3Amail.python.org+searching+through+mailing+list+archives -- Steve From cs at cskk.id.au Thu Feb 1 20:03:52 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 2 Feb 2018 12:03:52 +1100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: <015584fe-197b-4958-a623-64106c17ae2c@googlegroups.com> References: <015584fe-197b-4958-a623-64106c17ae2c@googlegroups.com> Message-ID: <20180202010352.GA10237@cskk.homeip.net> On 01Feb2018 15:48, jladasky at itu.edu wrote: >On Thursday, February 1, 2018 at 9:07:15 AM UTC-8, alister wrote: >> simple solution stop using google groups & use either the mailing list or >> a news server with an NNTP client > >Sigh. I've been on Usenet since 1986. Is this how it ends? I will move if I must. Might just mean you need to move from letting Google deliver it to you. The beauty of usenet is that taking down one provider doesn't break stuff. The Net interprets censorship as damage and routes around it. - John Gilmore >Let me ask those of you who are not using Google Groups: how do you search? >In my experience, searching through mailing list archives has been poor. Well, I cheat. When I join a mailing list I download its archives into my local mail store. The I can use whatever I like (currently notmuch) to index and search. Locally, even when offline. I appreciate that this doesn't work form random ad hoc mailing lists I don't use, and also that some mailing lists don't present downloadable archives (uncivilised IMO). Otherwise, there is DuckDuckGo and The Google. Cheers, Cameron Simpson (formerly cs at zip.com.au) From cs at cskk.id.au Thu Feb 1 20:07:53 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 2 Feb 2018 12:07:53 +1100 Subject: Trace back error just trying to run a simple import of requests In-Reply-To: References: Message-ID: <20180202010753.GA22785@cskk.homeip.net> On 01Feb2018 20:26, William Sewell wrote: >My python script which I run daily just blew up. So, I went into python to diagnose and just typed in the first line - import requests. I received screenfulls of errors. Why? What could have happened when nothing changed on my end? The traceback is copied below. > > >[cid:image001.png at 01D39B71.014E8850] Please followup to the list with a cut/paste of the error text. This list is text only and strips attachments as an aid to readability. We want to help you, but we deal in plain text - it is far more useful to all. Cheers, Cameron Simpson (formerly cs at zip.com.au) From william.sewell at wgu.edu Thu Feb 1 21:01:17 2018 From: william.sewell at wgu.edu (William Sewell) Date: Fri, 2 Feb 2018 02:01:17 +0000 Subject: Trace back error just trying to run a simple import of requests In-Reply-To: <20180202010753.GA22785@cskk.homeip.net> References: <20180202010753.GA22785@cskk.homeip.net> Message-ID: It is allright. I just cannot account for the sudden change. In my usual path everything worked fine. Now, I have to navigate to where the Python executable, even though I have added the path to the Windows path. Please close my case. Dr. William Sewell Direct Line: 385.428.5377 Toll Free: 877.435.7948 ext 5377 Course Instructor, IT WGU will be closed President's Day, February 19th. Mountain Time Office hours: Sun 3:00 PM - 8:00 PM, Mon 6:30 AM - 3:30 PM, Tue 9:30 AM - 6:30 PM, Wed 9:00 AM - 5:00 PM, Thu 6:30 AM - 3:30 PM Your success is very important to me. IT MSDA and Database Team cmdatabase at wgu.edu Toll Free: 1-877-435-7948 Student Services: 1-866-903-0110 WellConnect Counseling Services: 1-877-685-3269 Western Governors University mailto:william.sewell at wgu.edu More about WGU in Fast Company, CNN, NPR, NBC Nightly News, Money, The Atlantic, TIME, etc. wgu.edu -----Original Message----- From: Cameron Simpson [mailto:cs at cskk.id.au] Sent: Thursday, February 01, 2018 8:08 PM To: William Sewell Cc: python-list at python.org Subject: Re: Trace back error just trying to run a simple import of requests On 01Feb2018 20:26, William Sewell wrote: >My python script which I run daily just blew up. So, I went into python to diagnose and just typed in the first line - import requests. I received screenfulls of errors. Why? What could have happened when nothing changed on my end? The traceback is copied below. > > >[cid:image001.png at 01D39B71.014E8850] Please followup to the list with a cut/paste of the error text. This list is text only and strips attachments as an aid to readability. We want to help you, but we deal in plain text - it is far more useful to all. Cheers, Cameron Simpson (formerly cs at zip.com.au) From cs at cskk.id.au Thu Feb 1 22:40:39 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 2 Feb 2018 14:40:39 +1100 Subject: Trace back error just trying to run a simple import of requests In-Reply-To: References: Message-ID: <20180202034039.GA184@cskk.homeip.net> On 02Feb2018 02:01, William Sewell wrote: >It is allright. I just cannot account for the sudden change. In my usual path everything worked fine. Now, I have to navigate to where the Python executable, even though I have added the path to the Windows path. Unfortunately I'm not an expert in how Python works on Windows; I gather it is rather different an experience from my usual UNIX environment. I would guess that the requests module is not installed in a place in your normal Python's sys.path, but because the current directory is normally a part of that, navigating to the Python install area makes the module findable. You should be able to make it work anywhere with the right Python path setting. Hopefully a Windows person can assist with the specifics. >Please close my case. This is a mailing list! There's no ticket :-) Just idle people prepared to help. Cheers, Cameron Simpson (formerly cs at zip.com.au) > >Dr. William Sewell >Direct Line: 385.428.5377 Toll Free: 877.435.7948 ext 5377 >Course Instructor, IT > >WGU will be closed President's Day, February 19th. >Mountain Time Office hours: Sun 3:00 PM - 8:00 PM, Mon 6:30 AM - 3:30 PM, Tue 9:30 AM - 6:30 PM, Wed 9:00 AM - 5:00 PM, Thu 6:30 AM - 3:30 PM >Your success is very important to me. >IT MSDA and Database Team >cmdatabase at wgu.edu >Toll Free: 1-877-435-7948 >Student Services: 1-866-903-0110 >WellConnect Counseling Services: 1-877-685-3269 >Western Governors University >mailto:william.sewell at wgu.edu > > > >More about WGU in Fast Company, CNN, NPR, NBC Nightly News, Money, The Atlantic, TIME, etc. >wgu.edu > > >-----Original Message----- >From: Cameron Simpson [mailto:cs at cskk.id.au] >Sent: Thursday, February 01, 2018 8:08 PM >To: William Sewell >Cc: python-list at python.org >Subject: Re: Trace back error just trying to run a simple import of requests > >On 01Feb2018 20:26, William Sewell wrote: >>My python script which I run daily just blew up. So, I went into python to diagnose and just typed in the first line - import requests. I received screenfulls of errors. Why? What could have happened when nothing changed on my end? The traceback is copied below. >> >> >>[cid:image001.png at 01D39B71.014E8850] > >Please followup to the list with a cut/paste of the error text. This list is text only and strips attachments as an aid to readability. > >We want to help you, but we deal in plain text - it is far more useful to all. > >Cheers, >Cameron Simpson (formerly cs at zip.com.au) From dieter at handshake.de Fri Feb 2 02:06:19 2018 From: dieter at handshake.de (dieter) Date: Fri, 02 Feb 2018 08:06:19 +0100 Subject: Dependency injection: overriding defaults References: <874ln1yy91.fsf@handshake.de> <878tccjvz4.fsf@niobium.community.veritas.com> Message-ID: <87shaj98gk.fsf@handshake.de> Kushal Kumaran writes: > Victor Porton writes: >> dieter wrote: > ... >>> I would distinguish between the common library and distinct >>> applications (command line utility, daemon). The applications >>> configure the logging system (differently) while the library uses >>> uniform logging calls. >> >> You have essentially just repeated my requirements. >> >> But HOW to do this (using dependency_injector module)? >> > > No idea how dependency_injector affects this, but have you read the > logging howto? The section > https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library > would be appropriate. In fact, it was something like this I had in mind. From dieter at handshake.de Fri Feb 2 02:14:03 2018 From: dieter at handshake.de (dieter) Date: Fri, 02 Feb 2018 08:14:03 +0100 Subject: auto-correct a speech-to-text output and relate to of the words based on syllables References: <78b57c55-b57f-4e80-b390-5fb56df4012b@googlegroups.com> Message-ID: <87o9l7983o.fsf@handshake.de> naveen at emagevisionpl.com writes: > I have to make an application in which, > > The user asks a question, Google's API is used to convert the speech to text. But the problem is due to different accent the translator misunderstands the words. > I want my application to guess the word to the nearest word spoken by the user. > ... > But sometimes for more than a word it makes the same guess. > example: > The user speaks "Light". The system translates it as "Bright" > The user speaks "White" The system translates it as "Bright" As those words are phonetically quite apart (they have very different first consonants), some step in your processing chain does something seriously wrong. First thing to do: identify the responsible step. After that, one can try to find a way to improve on it. From dieter at handshake.de Fri Feb 2 02:24:36 2018 From: dieter at handshake.de (dieter) Date: Fri, 02 Feb 2018 08:24:36 +0100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <015584fe-197b-4958-a623-64106c17ae2c@googlegroups.com> Message-ID: <87k1vv97m3.fsf@handshake.de> jladasky at itu.edu writes: > ... > Let me ask those of you who are not using Google Groups: how do you search? In my experience, searching through mailing list archives has been poor. I am using "gmane.org" which has quite a good search (in my view). Not sure, whether "gmane.org" hosts "your" newsgroup. Of course, the service may go away in the future. From steve+comp.lang.python at pearwood.info Fri Feb 2 02:53:59 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 2 Feb 2018 07:53:59 +0000 (UTC) Subject: auto-correct a speech-to-text output and relate to of the words based on syllables References: <78b57c55-b57f-4e80-b390-5fb56df4012b@googlegroups.com> <87o9l7983o.fsf@handshake.de> Message-ID: On Fri, 02 Feb 2018 08:14:03 +0100, dieter wrote: >> The user speaks "Light". The system translates it as "Bright" The user >> speaks "White" The system translates it as "Bright" > > As those words are phonetically quite apart (they have very different > first consonants), some step in your processing chain does something > seriously wrong. I disagree: Light, Bright and White sound very similar. They're identical except for the first consonant: /la?t/ /b?a?t/ /wa?t/ and even those consonants sound very similar. Human beings can easily mishear or fail to distinguish between those words, e.g.: https://www.wordnik.com/words/we%20tripped%20a%20light%20fan%20dangle https://duckduckgo.com/?q=%22brighter+shade+of+pale%22+mondegreen (the name of the song is *Whiter* Shade of Pale, not "Lighter" or "Brighter"). We should not assume that the first consonant is always correct. Of course we would hope that a speech-to-text system would correctly match Light/Bright/White/Fright/etc but given the vagaries of human accents and pronunciation, we shouldn't be surprised if it sometimes gets them wrong. -- Steve From steve+comp.lang.python at pearwood.info Fri Feb 2 02:58:48 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 2 Feb 2018 07:58:48 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <015584fe-197b-4958-a623-64106c17ae2c@googlegroups.com> <87k1vv97m3.fsf@handshake.de> Message-ID: On Fri, 02 Feb 2018 08:24:36 +0100, dieter wrote: > jladasky at itu.edu writes: >> ... >> Let me ask those of you who are not using Google Groups: how do you >> search? In my experience, searching through mailing list archives has >> been poor. > > I am using "gmane.org" which has quite a good search (in my view). Are you sure? Have you tried it recently, and by recently I mean in the last year or so, since the host's hard drive melted down. As far as I am aware, gmane's web archives hasn't worked for many months. http://gmane.org/find.php?list=python http://home.gmane.org/ -- Steve From tkadm30 at yandex.com Fri Feb 2 04:17:35 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 2 Feb 2018 04:17:35 -0500 Subject: What is django-hotsauce? Message-ID: About Django-hotsauce: Scalable and high-performance WSGI microframework on top of Django and others: Django-hotsauce is the ultimate web development toolkit for rogue Python hackers and chronic weed users looking to build porn web sites in your mama basement! :) Typically used for advanced training of slackers, hackers, and unemployed people bored with life and looking to learn Schevo DBMS on PyPy. :) I think django-hotsauce is a great Python library for research/educational and experimental purpose. In specific, it is interesting to break Django ORM and use ZODB to develop your own models api... Anyways, have fun hacking django-hotsauce with PyPy. Hacking life is essential to happiness... :) Pragmatic hackers love to learn rogue ways to exploit Django api with JIT and PyPy. ;) cheers, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From cl at isbd.net Fri Feb 2 05:06:04 2018 From: cl at isbd.net (Chris Green) Date: Fri, 2 Feb 2018 10:06:04 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <015584fe-197b-4958-a623-64106c17ae2c@googlegroups.com> Message-ID: jladasky at itu.edu wrote: > On Thursday, February 1, 2018 at 9:07:15 AM UTC-8, alister wrote: > > > simple solution stop using google groups & use either the mailing list or > > a news server with an NNTP client > > Sigh. I've been on Usenet since 1986. Is this how it ends? I will move if I must. > > Let me ask those of you who are not using Google Groups: how do you search? > In my experience, searching through mailing list archives has been poor. > I read using NNTP. I run my own local newserver (leafnode, very simple to configure) that aggregates feeds from two public NNTP servers which provides some robustness if things go wrong with either. By configuring the time for which posts are retained by leafnode (to quite a long time) I can search back a long way. As I only read a few tens of newsgroups this doesn't require much disk space. -- Chris Green ? From arj.python at gmail.com Fri Feb 2 07:21:52 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Fri, 2 Feb 2018 16:21:52 +0400 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: i find the mail experience to be really nice and fool proof (except when some really bold spam makes it through) Abdur-Rahmaan Janhangeer https://abdurrahmaanjanhangeer.wordpress.com On 1 Feb 2018 21:09, "superchromix" wrote: > > Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, > started receiving spam messages several months ago. > > Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. > > When trying to access comp.lang.idl-pvwave, a message is now displayed, > stating that the group owner needs to remove the spam, and can then apply > to Google in order to have access reinstated. > > However, old public Usenet groups like this have no owner. The > comp.lang.idl-pvwave group is more than 20 years old. Hence, there is no > way to unblock the group. > > This is a serious problem, since the entire collection of postings going > back many years has been blocked, no just the spam. This resource is > frequently used by IDL programmers. > > Seeing the spam postings in this newsgroup, I expect something similar may > happen to comp.lang.python, soon. > -- > https://mail.python.org/mailman/listinfo/python-list > From abrault at mapgears.com Fri Feb 2 10:02:08 2018 From: abrault at mapgears.com (Alexandre Brault) Date: Fri, 2 Feb 2018 10:02:08 -0500 Subject: What is django-hotsauce? In-Reply-To: References: Message-ID: So it's a buzzword generator? Alex On 2018-02-02 04:17 AM, Etienne Robillard wrote: > About Django-hotsauce: > > Scalable and high-performance WSGI microframework on top of Django and > others: Django-hotsauce is the ultimate web development toolkit for > rogue Python hackers and chronic weed users looking to build porn web > sites in your mama basement! :) > > Typically used for advanced training of slackers, hackers, and > unemployed people bored with life and looking to learn Schevo DBMS on > PyPy. :) > > I think django-hotsauce is a great Python library for > research/educational and experimental purpose. > > In specific, it is interesting to break Django ORM and use ZODB to > develop your own models api... > > Anyways, have fun hacking django-hotsauce with PyPy. > > Hacking life is essential to happiness... :) > > Pragmatic hackers love to learn rogue ways to exploit Django api with > JIT and PyPy. ;) > > cheers, > > Etienne > > > > From grant.b.edwards at gmail.com Fri Feb 2 10:25:46 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 2 Feb 2018 15:25:46 +0000 (UTC) Subject: Trace back error just trying to run a simple import of requests References: <3a9fdd65-b4fb-bbdb-19e1-8a14a8199e1a@mrabarnett.plus.com> Message-ID: On 2018-02-01, MRAB wrote: > On 2018-02-01 22:32, Grant Edwards wrote: >> On 2018-02-01, William Sewell wrote: >> >>> My python script which I run daily just blew up. So, I went into >>> python to diagnose and just typed in the first line - import >>> requests. I received screenfulls of errors. Why? What could have >>> happened when nothing changed on my end? The traceback is copied >>> below. >>> >>> [cid:image001.png at 01D39B71.014E8850] >> >> I don't recoginze "cid:image001.png at 01D39B71.014E8850" as an error >> message. >> > Isn't it ironic that we've just had a thread about this very issue, > namely sending screenshots? I dunno, is it time for a long thread on what "ironic" means? ;) -- Grant Edwards grant.b.edwards Yow! ... I want to perform at cranial activities with gmail.com Tuesday Weld!! From grant.b.edwards at gmail.com Fri Feb 2 10:27:39 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 2 Feb 2018 15:27:39 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <015584fe-197b-4958-a623-64106c17ae2c@googlegroups.com> <87k1vv97m3.fsf@handshake.de> Message-ID: On 2018-02-02, dieter wrote: > jladasky at itu.edu writes: >> ... >> Let me ask those of you who are not using Google Groups: how do you search? In my experience, searching through mailing list archives has been poor. > > I am using "gmane.org" which has quite a good search (in my view). 1) Gmane's search sucked. [Using Google to search the Gmane site worked fairly well.] 2) Gmane's search and webui has been absent for years. -- Grant Edwards grant.b.edwards Yow! ... If I had heart at failure right now, gmail.com I couldn't be a more fortunate man!! From rosuav at gmail.com Fri Feb 2 10:34:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 3 Feb 2018 02:34:08 +1100 Subject: Trace back error just trying to run a simple import of requests In-Reply-To: References: <3a9fdd65-b4fb-bbdb-19e1-8a14a8199e1a@mrabarnett.plus.com> Message-ID: On Sat, Feb 3, 2018 at 2:25 AM, Grant Edwards wrote: > On 2018-02-01, MRAB wrote: >> On 2018-02-01 22:32, Grant Edwards wrote: >>> On 2018-02-01, William Sewell wrote: >>> >>>> My python script which I run daily just blew up. So, I went into >>>> python to diagnose and just typed in the first line - import >>>> requests. I received screenfulls of errors. Why? What could have >>>> happened when nothing changed on my end? The traceback is copied >>>> below. >>>> >>>> [cid:image001.png at 01D39B71.014E8850] >>> >>> I don't recoginze "cid:image001.png at 01D39B71.014E8850" as an error >>> message. >>> >> Isn't it ironic that we've just had a thread about this very issue, >> namely sending screenshots? > > I dunno, is it time for a long thread on what "ironic" means? ;) > An ironic thread is slightly lighter than a cobaltic thread, slightly heavier than a maganetic thread. It's also heavier than Chromium, which is quite an accomplishment, considering the atomic RAM weight of Chromium. ChrisA From tkadm30 at yandex.com Fri Feb 2 11:00:54 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 2 Feb 2018 11:00:54 -0500 Subject: What is django-hotsauce? In-Reply-To: References: Message-ID: <274d63f2-2bb7-ffbe-33d2-af627928723a@yandex.com> More like a platform to impress your cat with your JIT-powered web app :) Etienne Le 2018-02-02 ? 10:02, Alexandre Brault a ?crit?: > So it's a buzzword generator? > > Alex > > > On 2018-02-02 04:17 AM, Etienne Robillard wrote: >> About Django-hotsauce: >> >> Scalable and high-performance WSGI microframework on top of Django and >> others: Django-hotsauce is the ultimate web development toolkit for >> rogue Python hackers and chronic weed users looking to build porn web >> sites in your mama basement! :) >> >> Typically used for advanced training of slackers, hackers, and >> unemployed people bored with life and looking to learn Schevo DBMS on >> PyPy. :) >> >> I think django-hotsauce is a great Python library for >> research/educational and experimental purpose. >> >> In specific, it is interesting to break Django ORM and use ZODB to >> develop your own models api... >> >> Anyways, have fun hacking django-hotsauce with PyPy. >> >> Hacking life is essential to happiness... :) >> >> Pragmatic hackers love to learn rogue ways to exploit Django api with >> JIT and PyPy. ;) >> >> cheers, >> >> Etienne >> >> >> >> -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From python.list at tim.thechases.com Fri Feb 2 14:10:24 2018 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 2 Feb 2018 13:10:24 -0600 Subject: virtualenvwrapper under OpenBSD's ksh Message-ID: <20180202131024.24378879@bigbox.christie.dr> Under a new user account with ksh (the default) as the shell I issued the following: $ pip3 install --user virtualenvwrapper Successfully installed pbr-3.1.1 six-1.11.0 stevedore-1.28.0 virtualenv-clone-0.2.6 virtualenvwrapper-4.8.2 $ export WORKON_HOME=~/code/virtualenvs $ mkdir -p $WORKON_HOME Good so far. Based on https://bitbucket.org/dhellmann/virtualenvwrapper-hg it sounds like ksh should be supported. However when I try to enable it, I get: $ . ~/.local/bin/virtualenvwrapper.sh ksh: /home/tim/.local/bin/virtualenvwrapper.sh[97]: ${.sh.file}": bad substitution The line in question reads virtualenvwrapper.sh: export VIRTUALENVWRAPPER_SCRIPT="${.sh.file}" though it's not present in the latest tip version of the source. I tried pulling in that one virtualenvwrapper.sh file from the tip to see if that would remedy the issue but it complains $ . ~/tmp/virtualenvwrapper.sh ksh: /home/tim/tmp/virtualenvwrapper.sh[247]: syntax error: `(' unexpected on this line COMPREPLY=( $(compgen -W "`virtualenvwrapper_show_workon_options`" -- ${cur}) ) Is there something I'm missing or need to do to get pip (pip3.6) to pull in a working version of virtualenvwrapper for ksh? Thanks, -tkc From dieter at handshake.de Sat Feb 3 03:34:57 2018 From: dieter at handshake.de (dieter) Date: Sat, 03 Feb 2018 09:34:57 +0100 Subject: auto-correct a speech-to-text output and relate to of the words based on syllables References: <78b57c55-b57f-4e80-b390-5fb56df4012b@googlegroups.com> <87o9l7983o.fsf@handshake.de> Message-ID: <87372icvym.fsf@handshake.de> Steven D'Aprano writes: > On Fri, 02 Feb 2018 08:14:03 +0100, dieter wrote: > >>> The user speaks "Light". The system translates it as "Bright" The user >>> speaks "White" The system translates it as "Bright" >> >> As those words are phonetically quite apart (they have very different >> first consonants), some step in your processing chain does something >> seriously wrong. > > I disagree: Light, Bright and White sound very similar. They're identical > except for the first consonant: We have here an example that the first consonant can significantly influence the meaning. As a consequence, it will in general be spoken and affect the sound. And obviously, I should not be ignored when one is interested in a narrow match. From tkadm30 at yandex.com Sat Feb 3 05:24:41 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Sat, 3 Feb 2018 05:24:41 -0500 Subject: What is django-hotsauce? In-Reply-To: References: Message-ID: <2e549060-d5ea-5777-f88e-8137499706fb@yandex.com> Django-hotsauce is a Python-powered library for pragmatic hackers looking to exploit django with JIT. A commercial edition is also available for pre-order: https://www.livestore.ca/product/django-hotsauce/ Cheers, Etienne Le 2018-02-02 ? 10:02, Alexandre Brault a ?crit?: > So it's a buzzword generator? > > Alex > > > On 2018-02-02 04:17 AM, Etienne Robillard wrote: >> About Django-hotsauce: >> >> Scalable and high-performance WSGI microframework on top of Django and >> others: Django-hotsauce is the ultimate web development toolkit for >> rogue Python hackers and chronic weed users looking to build porn web >> sites in your mama basement! :) >> >> Typically used for advanced training of slackers, hackers, and >> unemployed people bored with life and looking to learn Schevo DBMS on >> PyPy. :) >> >> I think django-hotsauce is a great Python library for >> research/educational and experimental purpose. >> >> In specific, it is interesting to break Django ORM and use ZODB to >> develop your own models api... >> >> Anyways, have fun hacking django-hotsauce with PyPy. >> >> Hacking life is essential to happiness... :) >> >> Pragmatic hackers love to learn rogue ways to exploit Django api with >> JIT and PyPy. ;) >> >> cheers, >> >> Etienne >> >> >> >> -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From hjp-python at hjp.at Sat Feb 3 06:00:34 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 3 Feb 2018 12:00:34 +0100 Subject: New to Python and understanding problem In-Reply-To: References: Message-ID: <20180203110034.huxwpf7md6qyylj4@hjp.at> On 2018-01-29 19:14:57 +0100, Michelle Konzack wrote: > Am 2018-01-29 hackte Dan Stromberg in die Tasten: > > I don't see blueman on pypi, so this is probably part of the package > > you downloaded, and not something you need to "pip3 install". > > I have Python 2.7 and 3.5 from the Debian GNU/Linux repository installed > I use the Stable (Stretch) version. > > Installing blueman 2.0.4 (for Python 2.7) ended with a SEGFAULT. How did you install this? When I install blueman-2.0.4-1 from the Debian Stretch (i386) apt repository, it uses Python3: hjp 16620 2.0 0.5 110192 44316 pts/24 Sl+ 11:40 0:00 /usr/bin/python3 /usr/bin/blueman-applet And it seems to work (the icon shows up in the task bar and I can open the menu. I don't have any bluetooth hardware on my desktop, so I can't test that). A segfault is probably caused by a library, not by Python. If you got blueman from the Debian repository, report the problem at https://bugs.debian.org > Hence I downloaded the sources 2.1~alpha2 from the Experimantal Mirror, > comiled and packed it as Backport. > > Anything went fine, except, if I execute /usr/bin/blueman-applet it > does not find the path to blueman which is in > > /usr/lib/python-3.5/site-packages > > as subdirectory "bluman". This looks like a packaging error. AFAICS python3 packages on Debian are normally installed directly under /usr/lib/python3.5, not under /usr/lib/python-3.5/site-packages or /usr/lib/python-3.5/dist-packages (although the latter is on sys.path). Unless you have modified this yourself, please report it to Debian. hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From eryksun at gmail.com Sat Feb 3 06:09:52 2018 From: eryksun at gmail.com (eryk sun) Date: Sat, 3 Feb 2018 11:09:52 +0000 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> Message-ID: On Fri, Feb 2, 2018 at 6:28 PM, Gilmeh Serda wrote: > > M$'s excuse for a real Terminal, "Power" Shell (sigh), is _slightly_ > better but still lacking lots of features. Like a decent scripting > language. I loath VBS. ??,,, /puke PowerShell is a .NET scripting language that's available in Windows, Linux, and MacOS. VBScript is an unrelated scripting language that was more commonly used on Windows before PowerShell came along. On Windows, powershell.exe either allocates a console or inherits one from its parent. The Windows console adapts the message-based desktop environment to support command-line applications. It provides an input stream with keyboard and mouse records, and screen buffers that support UCS-2 and a 16-color palette. In Windows 10, it also functions as a virtual terminal with 24-bit color. In Windows 7 and up, each console is hosted by a separate instance of conhost.exe. Previously, NT systems hosted consoles in the system process, csrss.exe. In Windows 8 and up, the console API uses the ConDrv device for IPC with an attached console. Previously, NT systems used LPC ports and shared memory for this. In Windows 9x, the console window was hosted by conagent.exe, and the API used the VCOND device for IPC. Despite the ever-changing implementation details, the console API itself has been relatively stable for about 25 years. > It does have mouse selection (but still treats the text as a block of > characters, like a DOS box, and not lines of text, like Terminal) In Windows 10, the console uses line-wrapping selection by default, and rectangle selection requires holding ALT. From hjp-python at hjp.at Sat Feb 3 06:11:51 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 3 Feb 2018 12:11:51 +0100 Subject: 2.6.7: Does socket.gethostbyaddr truncate? In-Reply-To: References: <14988d6b-5039-029c-3295-5bb48479b5cf@vub.be> Message-ID: <20180203111151.kii2uwjmwtkrgx33@hjp.at> On 2018-01-30 08:56:16 -0800, Dan Stromberg wrote: > dig -x should return a single PTR in all cases, shouldn't it? No. dig -x should return *all* PTR records. There is usually at most one of them, but there may be several. (46 seems a bit much, but there really isn't any limit). > What IP are you using? Yup. I want to see an address with 46 PTR records, too ;-). > On Tue, Jan 30, 2018 at 4:05 AM, Antoon Pardon wrote: > > I am using python 2.6.7 to do a little network programming, but it seems I don't > > get all the results. > > > > When I call socket.gethostbyaddr(IP) entry [1] of the result is a list of 34 addresses. gethostbyaddr just calls the underlying C library function. It is possibly that this has a limit (either on the number of names or more likely on the packet size). hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sat Feb 3 07:09:40 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 3 Feb 2018 13:09:40 +0100 Subject: auto-correct a speech-to-text output and relate to of the words based on syllables In-Reply-To: <87372icvym.fsf@handshake.de> References: <78b57c55-b57f-4e80-b390-5fb56df4012b@googlegroups.com> <87o9l7983o.fsf@handshake.de> <87372icvym.fsf@handshake.de> Message-ID: <20180203120940.6jqtr6s7qr7mlhkx@hjp.at> On 2018-02-03 09:34:57 +0100, dieter wrote: > Steven D'Aprano writes: > > On Fri, 02 Feb 2018 08:14:03 +0100, dieter wrote: > >>> The user speaks "Light". The system translates it as "Bright" The user > >>> speaks "White" The system translates it as "Bright" > >> > >> As those words are phonetically quite apart (they have very different > >> first consonants), some step in your processing chain does something > >> seriously wrong. > > > > I disagree: Light, Bright and White sound very similar. They're identical > > except for the first consonant: > > We have here an example that the first consonant can significantly > influence the meaning. > As a consequence, it will in general be spoken and affect the sound. It will affect the sound, but not the same for every speaker. Since you have a German mail address, consider how differently "Chemie" is pronounced in German depending on which part of Germany you are from. Now consider someone speaking with a French or Indian or Chinese accent. Different languages have different phonemes, and humans generally learn to distinguish between them (and to disregard variances) in the first years. > And obviously, I should not be ignored when one is interested in > a narrow match. The difficulty is to *recognise* it correctly. Was that tangle of sound waves an "l" or an "r"? This not as unambiguous as you seem to think. So a speech-to-text program may hear "right" when the speaker was really saying "light". If you have only the output from that program you must determine whether "right" is correct or must be corrected to "light". hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From breamoreboy at gmail.com Sat Feb 3 07:33:36 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 3 Feb 2018 04:33:36 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: On Thursday, February 1, 2018 at 5:01:58 PM UTC, superchromix wrote: > Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, started receiving spam messages several months ago. > > Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. > > When trying to access comp.lang.idl-pvwave, a message is now displayed, stating that the group owner needs to remove the spam, and can then apply to Google in order to have access reinstated. > > However, old public Usenet groups like this have no owner. The comp.lang.idl-pvwave group is more than 20 years old. Hence, there is no way to unblock the group. > > This is a serious problem, since the entire collection of postings going back many years has been blocked, no just the spam. This resource is frequently used by IDL programmers. > > Seeing the spam postings in this newsgroup, I expect something similar may happen to comp.lang.python, soon. That's no big deal as this list used to have a wonderful signal to noise ratio, now it would be better to report the noise to signal ratio. I've just finished flagging up 45 consecutive messages on the group as spam, all of them from today and all of them from the "Case Solutions" crew. The downside of this list being blocked is that you'd all no doubt miss my wonderful contributions :) Unless of course a new list is opened, properly moderated, with a really original name like python-users. -- Kindest regards. Mark Lawrence. From linux4michelle at tamay-dogan.net Sat Feb 3 08:09:44 2018 From: linux4michelle at tamay-dogan.net (Michelle Konzack) Date: Sat, 3 Feb 2018 14:09:44 +0100 Subject: New to Python and understanding problem In-Reply-To: <20180203110034.huxwpf7md6qyylj4@hjp.at> References: <20180203110034.huxwpf7md6qyylj4@hjp.at> Message-ID: <76ce140c3cdeea6ebbb5f15306bbf659.squirrel@webmail.tamay-dogan.net> Hello peter and *, Am 2018-02-03 hackte Peter J. Holzer in die Tasten: > On 2018-01-29 19:14:57 +0100, Michelle Konzack wrote: >> Am 2018-01-29 hackte Dan Stromberg in die Tasten: >> > I don't see blueman on pypi, so this is probably part of the package >> > you downloaded, and not something you need to "pip3 install". >> >> I have Python 2.7 and 3.5 from the Debian GNU/Linux repository installed >> I use the Stable (Stretch) version. >> >> Installing blueman 2.0.4 (for Python 2.7) ended with a SEGFAULT. > > How did you install this? When I install blueman-2.0.4-1 from the Debian > Stretch (i386) apt repository, it uses Python3: > > hjp 16620 2.0 0.5 110192 44316 pts/24 Sl+ 11:40 0:00 > /usr/bin/python3 /usr/bin/blueman-applet > > And it seems to work (the icon shows up in the task bar and I can open > the menu. I don't have any bluetooth hardware on my desktop, so I can't > test that). > > A segfault is probably caused by a library, not by Python. If you got > blueman from the Debian repository, report the problem at > https://bugs.debian.org > >> Hence I downloaded the sources 2.1~alpha2 from the Experimantal Mirror, >> comiled and packed it as Backport. >> >> Anything went fine, except, if I execute /usr/bin/blueman-applet it >> does not find the path to blueman which is in >> >> /usr/lib/python-3.5/site-packages >> >> as subdirectory "bluman". > > This looks like a packaging error. AFAICS python3 packages on Debian are > normally installed directly under /usr/lib/python3.5, not under > /usr/lib/python-3.5/site-packages or /usr/lib/python-3.5/dist-packages > (although the latter is on sys.path). Unless you have modified this > yourself, please report it to Debian. I do not know what was in the experimental sources, but it seems a make clean has removed something and when I recompiled the package, it was correctly working. However, I use xorg, fvwm, trayer, blueman-applet like pasystray and was searchin the whole internet to solv this problem. Had installed and deinstalled packages and now it is magicaly working. Now I have to figure out, which of the packages pulled a missing dependency into the system. It is sad, that only monsters like KGE (it would install 1,8GByte on my system) and GNOME have all the right dependencies. But if you install something like debian base alsa xorg wdm fvwm vlc trayer blueman pasystray nothing is working! I think the above packages should install a working system! Now I have to figure out, WHY ALSA is working (pasystray show it) but there is no sound coming out of the speakers. I say only: "Welcome to the dependeny hell!" Thanks in avance -- Michelle Konzack Miila ITSystems @ TDnet GNU/Linux Developer 00372-54541400 From phamp at mindspring.com Sat Feb 3 11:41:07 2018 From: phamp at mindspring.com (pyotr filipivich) Date: Sat, 03 Feb 2018 08:41:07 -0800 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: breamoreboy at gmail.com on Sat, 3 Feb 2018 04:33:36 -0800 (PST) typed in comp.lang.python the following: >On Thursday, February 1, 2018 at 5:01:58 PM UTC, superchromix wrote: >> Our own programming discussion newsgroup, located at comp.lang.idl-pvwave, started receiving spam messages several months ago. >> >> Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google Groups. >> >> When trying to access comp.lang.idl-pvwave, a message is now displayed, stating that the group owner needs to remove the spam, and can then apply to Google in order to have access reinstated. >> >> However, old public Usenet groups like this have no owner. The comp.lang.idl-pvwave group is more than 20 years old. Hence, there is no way to unblock the group. >> >> This is a serious problem, since the entire collection of postings going back many years has been blocked, no just the spam. This resource is frequently used by IDL programmers. >> >> Seeing the spam postings in this newsgroup, I expect something similar may happen to comp.lang.python, soon. > >That's no big deal as this list used to have a wonderful signal to noise ratio, now it would be better to report the noise to signal ratio. I've just finished flagging up 45 consecutive messages on the group as spam, all of them from today and all of them from the "Case Solutions" crew. So you just indicated that of N postings, 45 different ones were spam? Rather than indicate that one poster of N is a spammer? I know, Google doesn't let you filter posts on author, and make specific authors or subjects to be ignored. And their search engine is a joke. >The downside of this list being blocked is that you'd all no doubt miss my wonderful contributions :) Unless of course a new list is opened, properly moderated, with a really original name like python-users. Those of us who do not use google-groups may not notice the loss of the google groupies. -- pyotr filipivich Next month's Panel: Graft - Boon or blessing? From hjp-python at hjp.at Sat Feb 3 12:56:32 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 3 Feb 2018 18:56:32 +0100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> Message-ID: <20180203175632.u2arrszbl6kksgp6@hjp.at> On 2018-02-03 04:33:36 -0800, breamoreboy at gmail.com wrote: > On Thursday, February 1, 2018 at 5:01:58 PM UTC, superchromix wrote: > > Our own programming discussion newsgroup, located at > > comp.lang.idl-pvwave, started receiving spam messages several months > > ago. > > > > Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google > > Groups. [...] > > Seeing the spam postings in this newsgroup, I expect something > > similar may happen to comp.lang.python, soon. > > That's no big deal as this list used to have a wonderful signal to > noise ratio, now it would be better to report the noise to signal > ratio. I've just finished flagging up 45 consecutive messages on the > group as spam, all of them from today and all of them from the "Case > Solutions" crew. > > The downside of this list being blocked is that you'd all no doubt > miss my wonderful contributions :) Unless of course a new list is > opened, properly moderated, with a really original name like > python-users. You seem to confuse the mailing-list and the newsgroup. The mailing-list doesn't have a spam problem, and it is already (lightly) moderated. The newsgroup does have a spam problem (as well as a few other problems, like gmane mangling message-ids and breaking threads). Google groups is an interface to the newsgroup. There is a bi-directional gateway between them, but they aren't the same thing. hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From kevindoney5 at gmail.com Sat Feb 3 14:10:07 2018 From: kevindoney5 at gmail.com (Kevin Doney) Date: Sat, 3 Feb 2018 14:10:07 -0500 Subject: Fwd: Syntax error In-Reply-To: References: Message-ID: Hi. *pip3 install --upgrade tensorflow-gpu* When I try the above command I get SyntaxError: invalid syntax Please help. From breamoreboy at gmail.com Sat Feb 3 14:34:53 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Sat, 3 Feb 2018 19:34:53 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: <20180203175632.u2arrszbl6kksgp6@hjp.at> References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <20180203175632.u2arrszbl6kksgp6@hjp.at> Message-ID: On 03/02/18 17:56, Peter J. Holzer wrote: > On 2018-02-03 04:33:36 -0800, breamoreboy at gmail.com wrote: >> On Thursday, February 1, 2018 at 5:01:58 PM UTC, superchromix wrote: >>> Our own programming discussion newsgroup, located at >>> comp.lang.idl-pvwave, started receiving spam messages several months >>> ago. >>> >>> Two weeks ago, access to comp.lang.idl-pvwave was blocked by Google >>> Groups. > [...] >>> Seeing the spam postings in this newsgroup, I expect something >>> similar may happen to comp.lang.python, soon. >> >> That's no big deal as this list used to have a wonderful signal to >> noise ratio, now it would be better to report the noise to signal >> ratio. I've just finished flagging up 45 consecutive messages on the >> group as spam, all of them from today and all of them from the "Case >> Solutions" crew. >> >> The downside of this list being blocked is that you'd all no doubt >> miss my wonderful contributions :) Unless of course a new list is >> opened, properly moderated, with a really original name like >> python-users. > > You seem to confuse the mailing-list and the newsgroup. The mailing-list > doesn't have a spam problem, and it is already (lightly) moderated. > > The newsgroup does have a spam problem (as well as a few other problems, > like gmane mangling message-ids and breaking threads). Google groups is > an interface to the newsgroup. > > There is a bi-directional gateway between them, but they aren't the same > thing. > > hp > No, they are one and the same thing, except that this is automatically blocked by the numpty moderators, whereas some things that I state on gg are passed, other things aren't. It is quite clear that the moderators are biased against people such as myself who are autistic My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From tjreedy at udel.edu Sat Feb 3 14:40:38 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 3 Feb 2018 14:40:38 -0500 Subject: Fwd: Syntax error In-Reply-To: References: Message-ID: On 2/3/2018 2:10 PM, Kevin Doney wrote: > Hi. > > *pip3 install --upgrade tensorflow-gpu* > > When I try the above command I get > SyntaxError: invalid syntax > > > Please help. This group helps those who help the group -- by asking questions with sufficient information to answer. Post the ENTIRE trackback. -- Terry Jan Reedy From rosuav at gmail.com Sat Feb 3 14:49:57 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 4 Feb 2018 06:49:57 +1100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups In-Reply-To: References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <20180203175632.u2arrszbl6kksgp6@hjp.at> Message-ID: On Sun, Feb 4, 2018 at 6:34 AM, Mark Lawrence wrote: > On 03/02/18 17:56, Peter J. Holzer wrote: >> You seem to confuse the mailing-list and the newsgroup. The mailing-list >> doesn't have a spam problem, and it is already (lightly) moderated. >> >> The newsgroup does have a spam problem (as well as a few other problems, >> like gmane mangling message-ids and breaking threads). Google groups is >> an interface to the newsgroup. >> >> There is a bi-directional gateway between them, but they aren't the same >> thing. >> >> hp >> > > No, they are one and the same thing, except that this is automatically > blocked by the numpty moderators, whereas some things that I state on gg are > passed, other things aren't. It is quite clear that the moderators are > biased against people such as myself who are autistic No, the moderators are biased against people who are constantly rude. Did you know that, autistic or not, you have the power to choose the tone of the words you type? ChrisA From alister.ware at ntlworld.com Sat Feb 3 15:28:15 2018 From: alister.ware at ntlworld.com (alister) Date: Sat, 03 Feb 2018 20:28:15 GMT Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <20180203175632.u2arrszbl6kksgp6@hjp.at> Message-ID: On Sun, 04 Feb 2018 06:49:57 +1100, Chris Angelico wrote: > On Sun, Feb 4, 2018 at 6:34 AM, Mark Lawrence > wrote: >> On 03/02/18 17:56, Peter J. Holzer wrote: >>> You seem to confuse the mailing-list and the newsgroup. The >>> mailing-list doesn't have a spam problem, and it is already (lightly) >>> moderated. >>> >>> The newsgroup does have a spam problem (as well as a few other >>> problems, >>> like gmane mangling message-ids and breaking threads). Google groups >>> is an interface to the newsgroup. >>> >>> There is a bi-directional gateway between them, but they aren't the >>> same thing. >>> >>> hp >>> >>> >> No, they are one and the same thing, except that this is automatically >> blocked by the numpty moderators, whereas some things that I state on >> gg are passed, other things aren't. It is quite clear that the >> moderators are biased against people such as myself who are autistic > > No, the moderators are biased against people who are constantly rude. > Did you know that, autistic or not, you have the power to choose the > tone of the words you type? > > ChrisA indeed my son is mildly autistic. I always explained to him that that diagnosis did not give him an excuse, it simply gave him an explanation of why things were harder for him. -- If it's worth doing, it's worth doing for money. From bgailer at gmail.com Sat Feb 3 15:38:29 2018 From: bgailer at gmail.com (bob gailer) Date: Sat, 3 Feb 2018 15:38:29 -0500 Subject: Fwd: Syntax error In-Reply-To: References: Message-ID: <45c39a47-d7e7-352a-2d9b-359c5cb72242@gmail.com> On 2/3/2018 2:40 PM, Terry Reedy wrote: > On 2/3/2018 2:10 PM, Kevin Doney wrote: >> Hi. >> >> *pip3 install --upgrade tensorflow-gpu* >> >> When I try the above command I get >> SyntaxError: invalid syntax >> >> >> Please help. > > This group helps those who help the group -- by asking questions with > sufficient information to answer.? Post the ENTIRE trackback. It also helps us to know: operating system (e.g., windows 10) what does "when I try" mean? (e.g. at a windows command prompt I entered ...) What the *'s are for best is to copy the entire session and paste it into your email. Example Microsoft Windows [Version 10.0.16299.192] (c) 2017 Microsoft Corporation. All rights reserved. C:\Users\bgailer>pip install foo Collecting foo ? Could not find a version that satisfies the requirement foo (from versions: ) No matching distribution found for foo We could presume in your case that you tried to enter your pip command in a python interactive session. For example: C:\Users\bgailer>python Python 3.3.5 (v3.3.5:62cf4e77f785, Mar? 9 2014, 10:35:05) [MSC v.1600 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> pip install foo ? File "", line 1 ??? pip install foo ????????????? ^ SyntaxError: invalid syntax >>> Why did you get that response? Because pip is not a python statement (python does not have commands); it is an executable program. So you need to use a command prompt or terminal. From tjreedy at udel.edu Sat Feb 3 17:28:48 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 3 Feb 2018 17:28:48 -0500 Subject: Fwd: Syntax error In-Reply-To: <45c39a47-d7e7-352a-2d9b-359c5cb72242@gmail.com> References: <45c39a47-d7e7-352a-2d9b-359c5cb72242@gmail.com> Message-ID: On 2/3/2018 3:38 PM, bob gailer wrote: > On 2/3/2018 2:40 PM, Terry Reedy wrote: >> On 2/3/2018 2:10 PM, Kevin Doney wrote: >>> Hi. >>> >>> *pip3 install --upgrade tensorflow-gpu* >>> >>> When I try the above command I get >>> SyntaxError: invalid syntax >>> >>> >>> Please help. >> >> This group helps those who help the group -- by asking questions with >> sufficient information to answer.? Post the ENTIRE trackback. > It also helps us to know: > operating system (e.g., windows 10) > what does "when I try" mean? (e.g. at a windows command prompt I entered > ...) > What the *'s are for > best is to copy the entire session and paste it into your email. Example I entirely agree. > Microsoft Windows [Version 10.0.16299.192] > (c) 2017 Microsoft Corporation. All rights reserved. > > C:\Users\bgailer>pip install foo > Collecting foo > ? Could not find a version that satisfies the requirement foo (from > versions: ) > No matching distribution found for foo > > We could presume in your case that you tried to enter your pip command > in a python interactive session. This is a good guess, and I hope it is correct, because the alternative of a SyntaxError in some execution path in pip or dependency or python stdlib module or package being installed is worse. > For example: > > C:\Users\bgailer>python > Python 3.3.5 (v3.3.5:62cf4e77f785, Mar? 9 2014, 10:35:05) [MSC v.1600 64 > bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> pip install foo > ? File "", line 1 > ??? pip install foo > ????????????? ^ > SyntaxError: invalid syntax > >>> > > Why did you get that response? Because pip is not a python statement > (python does not have commands); it is an executable program. So you > need to use a command prompt or terminal. > > -- Terry Jan Reedy From cs at cskk.id.au Sat Feb 3 23:54:35 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 4 Feb 2018 15:54:35 +1100 Subject: Fwd: Syntax error In-Reply-To: References: Message-ID: <20180204045435.GA38451@cskk.homeip.net> On 03Feb2018 14:10, Kevin Doney wrote: >*pip3 install --upgrade tensorflow-gpu* > >When I try the above command I get >SyntaxError: invalid syntax > >Please help. It looks like you typed that command to the Python prompt, based on the distinctively Python "SyntaxError" response. That command is supposed to be typed to a shell prompt. It is not Python code, it is a command line to install a package. But as with the other responders, it is always helpful if you post the entire text, including the prompt you're using (which with show whether this was Python or a command shell etc) and if you've got a Python error, the full traceback. I suspect some people think this is noisy verbiage that would make their question hard to read, but it is usually vital context for the error message. Cheers, Cameron Simpson (formerly cs at zip.com.au) From steve+comp.lang.python at pearwood.info Sun Feb 4 04:58:58 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 4 Feb 2018 09:58:58 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Groups References: <5d3a5c5f-fe52-4514-b2d6-800c5b0cf647@googlegroups.com> <20180203175632.u2arrszbl6kksgp6@hjp.at> Message-ID: On Sun, 04 Feb 2018 06:49:57 +1100, Chris Angelico wrote: > On Sun, Feb 4, 2018 at 6:34 AM, Mark Lawrence > wrote: >> On 03/02/18 17:56, Peter J. Holzer wrote: >>> You seem to confuse the mailing-list and the newsgroup. The >>> mailing-list doesn't have a spam problem, and it is already (lightly) >>> moderated. >>> >>> The newsgroup does have a spam problem (as well as a few other >>> problems, like gmane mangling message-ids and breaking threads). >>> Google groups is an interface to the newsgroup. >>> >>> There is a bi-directional gateway between them, but they aren't the >>> same thing. >>> >>> hp >>> >>> >> No, they are one and the same thing, except that this is automatically >> blocked by the numpty moderators, whereas some things that I state on >> gg are passed, other things aren't. It is quite clear that the >> moderators are biased against people such as myself who are autistic > > No, the moderators are biased against people who are constantly rude. > Did you know that, autistic or not, you have the power to choose the > tone of the words you type? How about people with Tourette's Syndrome and a hair-trigger "Send" command? :-) Seriously though, Mark, the newsgroup and the mailing list aren't the same thing. They use different technology, they're run by different organisations, and they have different moderators (actual people moderating the mailing list, nobody moderating the newsgroup). You can see from the detailed message headers which messages reach you directly via the newsgroup and which have come through the mailing list, or vice versa. With a little care, you can even ensure that a message goes to the newgroup but not the mailing list, although I'm not sure if you can do it the other way. -- Steve From qingyun.tao at tophant.com Sun Feb 4 06:26:36 2018 From: qingyun.tao at tophant.com (=?utf-8?B?6Zm26Z2S5LqR?=) Date: Sun, 4 Feb 2018 19:26:36 +0800 Subject: Why not have a recvall method? In-Reply-To: <76ce140c3cdeea6ebbb5f15306bbf659.squirrel@webmail.tamay-dogan.net> References: <20180203110034.huxwpf7md6qyylj4@hjp.at> <76ce140c3cdeea6ebbb5f15306bbf659.squirrel@webmail.tamay-dogan.net> Message-ID: Hello, allThe socket object has a `sendall` method that can send all bytes you specified. Oppositely, socket only has a recv method. I wonder why there is not a `recvall` method? To workaround this, I use `f = socket.makefile('rb')`, then `call f.read(n)` Thanks. From shlyoko at gmail.com Sun Feb 4 07:47:08 2018 From: shlyoko at gmail.com (Emil Natan) Date: Sun, 4 Feb 2018 14:47:08 +0200 Subject: 2.6.7: Does socket.gethostbyaddr truncate? In-Reply-To: <20180203111151.kii2uwjmwtkrgx33@hjp.at> References: <14988d6b-5039-029c-3295-5bb48479b5cf@vub.be> <20180203111151.kii2uwjmwtkrgx33@hjp.at> Message-ID: On Sat, Feb 3, 2018 at 1:11 PM, Peter J. Holzer wrote: > On 2018-01-30 08:56:16 -0800, Dan Stromberg wrote: > > dig -x should return a single PTR in all cases, shouldn't it? > > No. dig -x should return *all* PTR records. There is usually at most one > of them, but there may be several. (46 seems a bit much, but there > really isn't any limit). > > > What IP are you using? > > Yup. I want to see an address with 46 PTR records, too ;-). > > > On Tue, Jan 30, 2018 at 4:05 AM, Antoon Pardon > wrote: > > > I am using python 2.6.7 to do a little network programming, but it > seems I don't > > > get all the results. > > > > > > When I call socket.gethostbyaddr(IP) entry [1] of the result is a list > of 34 addresses. > > gethostbyaddr just calls the underlying C library function. It is > possibly that this has a limit (either on the number of names or more > likely on the packet size). > > hp > > -- > _ | Peter J. Holzer | we build much bigger, better disasters now > |_|_) | | because we have much more sophisticated > | | | hjp at hjp.at | management tools. > __/ | http://www.hjp.at/ | -- Ross Anderson > > -- > https://mail.python.org/mailman/listinfo/python-list > > I would guess it's the packet size limitation. Using EDNS0 you can have packet size of up 4096 bytes, but it's the underlying IP protocol which limits further the amount of data in a single packet. Given the 20 bytes IPv4 header and 8 bytes DNS header, that leaves 1472 bytes for data. Bigger payload leads to IP fragmentation which in many cases is blocked by firewalls or other network devices on the way. In your case dig -x runs on the same machine where the Python code runs? If yes, then it's not network issue. What's the size of the response containing 34 addresses? What's the size of the response from dig when all 46 addresses are returned? Emil From qingyun.tao at tophant.com Sun Feb 4 08:26:16 2018 From: qingyun.tao at tophant.com (=?utf-8?B?6Zm26Z2S5LqR?=) Date: Sun, 4 Feb 2018 21:26:16 +0800 Subject: Why not have a recvall method? Message-ID: Hello, all The socket object has a `sendall` method that can send all bytes you specified. Oppositely, socket only has a recv method. I wonder why there is not a `recvall` method? To workaround this, I use `f = socket.makefile('rb')`, then all `f.read(n)` Thanks. From steve+comp.lang.python at pearwood.info Sun Feb 4 08:31:27 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 4 Feb 2018 13:31:27 +0000 (UTC) Subject: Why not have a recvall method? References: <20180203110034.huxwpf7md6qyylj4@hjp.at> <76ce140c3cdeea6ebbb5f15306bbf659.squirrel@webmail.tamay-dogan.net> Message-ID: On Sun, 04 Feb 2018 19:26:36 +0800, ??? wrote: > Hello, allThe socket object has a `sendall` method that can send all > bytes you specified. Oppositely, socket only has a recv method. I wonder > why there is not a `recvall` method? To workaround this, I use `f = > socket.makefile('rb')`, then `call f.read(n)` Thanks. I am not an expert on sockets, but since f.read(n) will read a maximum of n bytes, isn't that the same as socket_obj.recv(n)? -- Steve From qingyun.tao at tophant.com Sun Feb 4 09:01:24 2018 From: qingyun.tao at tophant.com (=?utf-8?B?6Zm26Z2S5LqR?=) Date: Sun, 4 Feb 2018 22:01:24 +0800 Subject: Why not have a recvall method? In-Reply-To: References: <20180203110034.huxwpf7md6qyylj4@hjp.at> <76ce140c3cdeea6ebbb5f15306bbf659.squirrel@webmail.tamay-dogan.net> Message-ID: makefile('rb') return a io.BufferedReader and the doc(https://docs.python.org/3/library/io.html#io.BufferedIOBase.read) says: If the argument is positive, and the underlying raw stream is not interactive, multiple raw reads may be issued to satisfy the byte count (unless EOF is reached first). ------------------ Original ------------------ From: "Steven D'Aprano"; Date: Sun, Feb 4, 2018 09:31 PM To: "python-list"; Subject: Re: Why not have a recvall method? On Sun, 04 Feb 2018 19:26:36 +0800, ??? wrote: > Hello, allThe socket object has a `sendall` method that can send all > bytes you specified. Oppositely, socket only has a recv method. I wonder > why there is not a `recvall` method? To workaround this, I use `f = > socket.makefile('rb')`, then `call f.read(n)` Thanks. I am not an expert on sockets, but since f.read(n) will read a maximum of n bytes, isn't that the same as socket_obj.recv(n)? -- Steve -- https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Sun Feb 4 09:09:44 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 5 Feb 2018 01:09:44 +1100 Subject: 2.6.7: Does socket.gethostbyaddr truncate? In-Reply-To: <14988d6b-5039-029c-3295-5bb48479b5cf@vub.be> References: <14988d6b-5039-029c-3295-5bb48479b5cf@vub.be> Message-ID: On Tue, Jan 30, 2018 at 11:05 PM, Antoon Pardon wrote: > I am using python 2.6.7 to do a little network programming, but it seems I don't > get all the results. > > When I call socket.gethostbyaddr(IP) entry [1] of the result is a list of 34 addresses. > > However when I use: dig -x IP I get a list of 46 addresses. > > Am I using the wrong function? Is this a bug? If the latter, is there a work around? > Can you post the full output of dig? Maybe it's switched to TCP. ChrisA From jqian at tibco.com Sun Feb 4 11:52:00 2018 From: jqian at tibco.com (Jason Qian) Date: Sun, 4 Feb 2018 11:52:00 -0500 Subject: Help on convert PyObject to string (c) Python 3.6 Message-ID: Hi, This is the case of calling python from c and the python function will return a string. It seems python been called correctly, but got error when convert the python string to c string. -- c -- PyObject* pValue = PyObject_CallObject(pFunc, pArgs); -- python -- import string, random def StringGen(argv): out_string_size=int(argv); output_data=''.join(random.choice(string.ascii_lowercase) for x in range(out_string_size)) return output_data I try: PyObject* pValue = PyObject_CallObject(pFunc, pArgs); const char* sp = 0; if (!PyArg_ParseTuple(pValue, "s", &sp)) { } and got "SystemError: new style getargs format but argument is not a tuple" Thanks for the help From rosuav at gmail.com Sun Feb 4 12:02:39 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 5 Feb 2018 04:02:39 +1100 Subject: Help on convert PyObject to string (c) Python 3.6 In-Reply-To: References: Message-ID: On Mon, Feb 5, 2018 at 3:52 AM, Jason Qian via Python-list wrote: > Hi, > > This is the case of calling python from c and the python function will > return a string. > > It seems python been called correctly, but got error when convert the > python string to c string. > > -- c -- > > PyObject* pValue = PyObject_CallObject(pFunc, pArgs); > > > -- python -- > > import string, random > def StringGen(argv): > out_string_size=int(argv); > output_data=''.join(random.choice(string.ascii_lowercase) for x in > range(out_string_size)) > return output_data > > > I try: > > PyObject* pValue = PyObject_CallObject(pFunc, pArgs); > const char* sp = 0; > if (!PyArg_ParseTuple(pValue, "s", &sp)) { > } > > and got > > "SystemError: new style getargs format but argument is not a tuple" > You're using something that is specifically for parsing *arguments* (as indicated by the "PyArg" prefix). If you want to get a 'const char *' from a Python string, you probably want to encode it UTF-8. There's a convenience function for that: https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AsUTF8AndSize And if you know for sure that there won't be any \0 in the string, you can use the next function in the docs, which doesn't bother returning the size. (It'll always be null-terminated.) ChrisA From jqian at tibco.com Sun Feb 4 14:16:10 2018 From: jqian at tibco.com (Jason Qian) Date: Sun, 4 Feb 2018 14:16:10 -0500 Subject: Help on convert PyObject to string (c) Python 3.6 In-Reply-To: References: Message-ID: Hi Chris, Thanks a lot ! Using PyUnicode_DecodeUTF8 fix the problem. On Sun, Feb 4, 2018 at 12:02 PM, Chris Angelico wrote: > On Mon, Feb 5, 2018 at 3:52 AM, Jason Qian via Python-list > wrote: > > Hi, > > > > This is the case of calling python from c and the python function > will > > return a string. > > > > It seems python been called correctly, but got error when convert the > > python string to c string. > > > > -- c -- > > > > PyObject* pValue = PyObject_CallObject(pFunc, pArgs); > > > > > > -- python -- > > > > import string, random > > def StringGen(argv): > > out_string_size=int(argv); > > output_data=''.join(random.choice(string.ascii_lowercase) for x in > > range(out_string_size)) > > return output_data > > > > > > I try: > > > > PyObject* pValue = PyObject_CallObject(pFunc, pArgs); > > const char* sp = 0; > > if (!PyArg_ParseTuple(pValue, "s", &sp)) { > > } > > > > and got > > > > "SystemError: new style getargs format but argument is not a tuple" > > > > You're using something that is specifically for parsing *arguments* > (as indicated by the "PyArg" prefix). If you want to get a 'const char > *' from a Python string, you probably want to encode it UTF-8. There's > a convenience function for that: > > https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AsUTF8AndSize > > And if you know for sure that there won't be any \0 in the string, you > can use the next function in the docs, which doesn't bother returning > the size. (It'll always be null-terminated.) > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From jqian at tibco.com Sun Feb 4 15:01:15 2018 From: jqian at tibco.com (Jason Qian) Date: Sun, 4 Feb 2018 15:01:15 -0500 Subject: Help on PyImport_Import(pNAme) Message-ID: Hi, This only works when loading modules from the current directory. Is there a way I can load from somewhere else ? Thanks for help, From robalspach3 at gmail.com Sun Feb 4 16:09:10 2018 From: robalspach3 at gmail.com (Rob Alspach) Date: Sun, 4 Feb 2018 16:09:10 -0500 Subject: 3.6.4 IDLE Message-ID: Hi Everybody! I just installed python 3.6.4 it was a fresh install (no former versions). I have never used python but have some IS background. The videos I am seeing online all show the IDE / IDLE; however, it does not install for me. I am installing from the download python-3.6.4-embed-amd64.zip. I am running windows 10. I also have Oracle on this laptop. The file does not have an installer and also does not write to the file path. I have already uninstalled and reinstalled the problem remains. I can update the path no problem but where can I get the IDLE / IDE to follow along with the videos on youtube? I am using notepad currently but have no way to execute any files I write. If I save a file in notepad and open through python the python window terminates when the operation is complete (much to fast for these old eyes). From hba008 at aol.com Sun Feb 4 16:37:38 2018 From: hba008 at aol.com (hba008 at aol.com) Date: Sun, 4 Feb 2018 16:37:38 -0500 Subject: Can't load Python program on my HP ENVY computer Message-ID: <16162c0cef9-c8f-8634@webjasstg-vab50.srv.aolmail.net> I have an HP ENVY TouchSmart 17 Notebook PC. Windows 8.1. Intel(R) Core(TM) i7-4700 mQ cpu @ 2.40 ghz 2.40ghz 64-bit operation system x64 based processor Full Windows touch support with 10 touch points Can you send me the link to the correct Python version that will run on this computer. I have tried to install Python several times and it just keeps hanging up in the initialization phase. Thanks, B From drsalists at gmail.com Sun Feb 4 17:01:13 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 4 Feb 2018 14:01:13 -0800 Subject: Why not have a recvall method? In-Reply-To: References: Message-ID: On Sun, Feb 4, 2018 at 5:26 AM, ??? wrote: > Hello, all > The socket object has a `sendall` method that can send all bytes you specified. > Oppositely, socket only has a recv method. I wonder why there is not a `recvall` method? > To workaround this, I use `f = socket.makefile('rb')`, then all `f.read(n)` > Thanks. You're probably good with socket.makefile('rb'). An alternative that allows things like reading null-terminated input: http://stromberg.dnsalias.org/~strombrg/bufsock.html I wrote it and have been using it in production for years. From dallasdisabilityattorney at gmail.com Sun Feb 4 17:26:10 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Sun, 4 Feb 2018 14:26:10 -0800 (PST) Subject: Parsing Nested List Message-ID: <489f288d-11b9-4b28-94d6-2ac7c14f49b6@googlegroups.com> I am trying to parse a Python nested list that is the result of the getOutlines() function of module PyPFD2 using pyparsing module. This is the result I get. what in the world are 'expandtabs' and why is that making a difference to my parse attempt? Python Code 7 import PPDF2,pyparsing from pyparsing import Word, alphas, nums pdfFileObj=open('x.pdf','rb') pdfReader=PyPDF2.PdfFileReader(pdfFileObj) List=pdfReader.getOutlines() myparser = Word( alphas ) + Word(nums, exact=2) +"of" + Word(nums, exact=2) myparser.parseString(List) This is the error I get: Traceback (most recent call last): File "", line 1, in myparser.parseString(List) File "C:\python\lib\site-packages\pyparsing.py", line 1620, in parseString instring = instring.expandtabs() AttributeError: 'list' object has no attribute 'expandtabs' Thanks so much, not getting any helpful responses from https://python-forum.io. From rosuav at gmail.com Sun Feb 4 17:36:04 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 5 Feb 2018 09:36:04 +1100 Subject: Parsing Nested List In-Reply-To: <489f288d-11b9-4b28-94d6-2ac7c14f49b6@googlegroups.com> References: <489f288d-11b9-4b28-94d6-2ac7c14f49b6@googlegroups.com> Message-ID: On Mon, Feb 5, 2018 at 9:26 AM, Stanley Denman wrote: > I am trying to parse a Python nested list that is the result of the getOutlines() function of module PyPFD2 using pyparsing module. This is the result I get. what in the world are 'expandtabs' and why is that making a difference to my parse attempt? > > Python Code > 7 > import PPDF2,pyparsing > from pyparsing import Word, alphas, nums > pdfFileObj=open('x.pdf','rb') > pdfReader=PyPDF2.PdfFileReader(pdfFileObj) > List=pdfReader.getOutlines() > myparser = Word( alphas ) + Word(nums, exact=2) +"of" + Word(nums, exact=2) > myparser.parseString(List) > > This is the error I get: > > Traceback (most recent call last): > File "", line 1, in > myparser.parseString(List) > File "C:\python\lib\site-packages\pyparsing.py", line 1620, in parseString > instring = instring.expandtabs() > AttributeError: 'list' object has no attribute 'expandtabs' > > Thanks so much, not getting any helpful responses from https://python-forum.io. By the look of this code, it's expecting a string. (The variable name "instring" is suggestive of this, and strings DO have an expandtabs method.) You're calling a method named "parseString", and presumably giving it a list. I don't know what you mean by "nested" though. Maybe you want to iterate over the list and parse each of the strings in it? More info about what you're trying to do here would help. ChrisA From steve+comp.lang.python at pearwood.info Sun Feb 4 18:03:25 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 4 Feb 2018 23:03:25 +0000 (UTC) Subject: Parsing Nested List References: <489f288d-11b9-4b28-94d6-2ac7c14f49b6@googlegroups.com> Message-ID: On Sun, 04 Feb 2018 14:26:10 -0800, Stanley Denman wrote: > I am trying to parse a Python nested list that is the result of the > getOutlines() function of module PyPFD2 using pyparsing module. pyparsing parses strings, not lists. I fear that you have completely misunderstood what pyparsing does: it isn't a general-purpose parser of arbitrary Python objects like lists. Like most parsers (actually, all parsers that I know of...) it takes text as input and produces some sort of machine representation: https://en.wikipedia.org/wiki/Parsing#Computer_languages So your code is not working because you are calling parseString() with a list argument: myparser.parseString(List) The name of the function, parseString(), should have been a hint that it requires a *string* as argument. You have generated an outline: List = pdfReader.getOutlines() but do you know what the format of that list is? I'm going to assume that it looks something like this: ['ABCD 01 of 99', 'EFGH 02 of 99', 'IJKL 03 of 99', ...] since that matches the template you gave to pyparsing. Notice that: - words are separated by spaces; - the first word is any arbitrary word, made up of just letters; - followed by EXACTLY two digits; - followed by the word "of"; - followed by EXACTLY two digits. Furthermore, I'm assuming it is a simple, non-nested list. If that is not the case, you will need to explain precisely what the format of the outline actually is. To parse this list is simple and pyparsing is not required: for item in List: words = item.split() if len(words) != 4: raise ValueError('bad input data: %r' % item) first, number, x, total = words number = int(number) assert x == 'of' total = int(total) print(first, number, total) Hope this helps. (Please keep any replies on the list.) -- Steve From rosuav at gmail.com Sun Feb 4 18:19:02 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 5 Feb 2018 10:19:02 +1100 Subject: 3.6.4 IDLE In-Reply-To: References: Message-ID: On Mon, Feb 5, 2018 at 8:09 AM, Rob Alspach wrote: > Hi Everybody! I just installed python 3.6.4 it was a fresh install (no > former versions). I have never used python but have some IS background. The > videos I am seeing online all show the IDE / IDLE; however, it does not > install for me. I am installing from the download > python-3.6.4-embed-amd64.zip. I am running windows 10. I also have Oracle > on this laptop. > > The file does not have an installer and also does not write to the file > path. I have already uninstalled and reinstalled the problem remains. I can > update the path no problem but where can I get the IDLE / IDE to follow > along with the videos on youtube? I am using notepad currently but have no > way to execute any files I write. If I save a file in notepad and open > through python the python window terminates when the operation is complete > (much to fast for these old eyes). What you'll normally want to do is get the regular installer. Go to python.org and look at the download links right there on the front page; that'll give you a much more normal way of getting hold of the full Python suite. ChrisA From dallasdisabilityattorney at gmail.com Sun Feb 4 18:32:37 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Sun, 4 Feb 2018 15:32:37 -0800 (PST) Subject: Parsing Nested List In-Reply-To: <489f288d-11b9-4b28-94d6-2ac7c14f49b6@googlegroups.com> References: <489f288d-11b9-4b28-94d6-2ac7c14f49b6@googlegroups.com> Message-ID: On Sunday, February 4, 2018 at 4:26:24 PM UTC-6, Stanley Denman wrote: > I am trying to parse a Python nested list that is the result of the getOutlines() function of module PyPFD2 using pyparsing module. This is the result I get. what in the world are 'expandtabs' and why is that making a difference to my parse attempt? > > Python Code > 7 > import PPDF2,pyparsing > from pyparsing import Word, alphas, nums > pdfFileObj=open('x.pdf','rb') > pdfReader=PyPDF2.PdfFileReader(pdfFileObj) > List=pdfReader.getOutlines() > myparser = Word( alphas ) + Word(nums, exact=2) +"of" + Word(nums, exact=2) > myparser.parseString(List) > > This is the error I get: > > Traceback (most recent call last): > File "", line 1, in > myparser.parseString(List) > File "C:\python\lib\site-packages\pyparsing.py", line 1620, in parseString > instring = instring.expandtabs() > AttributeError: 'list' object has no attribute 'expandtabs' > > Thanks so much, not getting any helpful responses from https://python-forum.io. From dallasdisabilityattorney at gmail.com Sun Feb 4 19:01:09 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Sun, 4 Feb 2018 16:01:09 -0800 (PST) Subject: Parsing Nested List In-Reply-To: References: <489f288d-11b9-4b28-94d6-2ac7c14f49b6@googlegroups.com> Message-ID: On Sunday, February 4, 2018 at 5:06:26 PM UTC-6, Steven D'Aprano wrote: > On Sun, 04 Feb 2018 14:26:10 -0800, Stanley Denman wrote: > > > I am trying to parse a Python nested list that is the result of the > > getOutlines() function of module PyPFD2 using pyparsing module. > > pyparsing parses strings, not lists. > > I fear that you have completely misunderstood what pyparsing does: it > isn't a general-purpose parser of arbitrary Python objects like lists. > Like most parsers (actually, all parsers that I know of...) it takes text > as input and produces some sort of machine representation: > > https://en.wikipedia.org/wiki/Parsing#Computer_languages > > > So your code is not working because you are calling parseString() with a > list argument: > > myparser.parseString(List) > > > The name of the function, parseString(), should have been a hint that it > requires a *string* as argument. > > You have generated an outline: > > List = pdfReader.getOutlines() > > but do you know what the format of that list is? I'm going to assume that > it looks something like this: > > ['ABCD 01 of 99', 'EFGH 02 of 99', 'IJKL 03 of 99', ...] > > since that matches the template you gave to pyparsing. Notice that: > > - words are separated by spaces; > > - the first word is any arbitrary word, made up of just letters; > > - followed by EXACTLY two digits; > > - followed by the word "of"; > > - followed by EXACTLY two digits. > > Furthermore, I'm assuming it is a simple, non-nested list. If that is not > the case, you will need to explain precisely what the format of the > outline actually is. > > To parse this list is simple and pyparsing is not required: > > for item in List: > words = item.split() > if len(words) != 4: > raise ValueError('bad input data: %r' % item) > first, number, x, total = words > number = int(number) > assert x == 'of' > total = int(total) > print(first, number, total) > > > > > Hope this helps. > > (Please keep any replies on the list.) > > > > -- > Steve Thank you so much Steve. I do seem to be barking up the wrong tree. The result of running getOutlines() is indeed a nested list: it is the pdfs bookmarks. There are 3 levels: level 1 is the section from A-F. When a section there are exhibits, so in Section A we have exhibits 1A to nA. Finally there are bookmarks for individual pages in an exhibit. So we have this for Section A: [{'/Title': 'Section A. Payment Documents/Decisions', '/Page': IndirectObject(1, 0), '/Type': '/FitB'}, [{'/Title': '1A: Disability Determination Transmittal (831) Dec. Dt.: 05/27/2016 (1 page)', '/Page': IndirectObject(1, 0), '/Type': '/FitB'}, [{'/Title': '1A (Page 1 of 1)', '/Page': IndirectObject(1, 0), '/Type': '/FitB'}], {'/Title': '2A: Disability Determination Explanation (DDE) Dec. Dt.: 05/27/2016 (10 pages)', '/Page': IndirectObject(6, 0), '/Type': '/FitB'}, [{'/Title': '2A (Page 1 of 10)', '/Page': IndirectObject(6, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 2 of 10)', '/Page': IndirectObject(10, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 3 of 10)', '/Page': IndirectObject(14, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 4 of 10)', '/Page': IndirectObject(18, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 5 of 10)', '/Page': IndirectObject(22, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 6 of 10)', '/Page': IndirectObject(26, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 7 of 10)', '/Page': IndirectObject(30, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 8 of 10)', '/Page': IndirectObject(34, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 9 of 10)', '/Page': IndirectObject(38, 0), '/Type': '/FitB'}, {'/Title': '2A (Page 10 of 10)', '/Page': IndirectObject(42, 0), '/Type': '/FitB'}], {'/Title': '3A: ALJ Hearing Decision (ALJDEC) Dec. Dt.: 12/17/2012 (22 pages)', '/Page': IndirectObject(47, 0), '/Type': '/FitB'}, [{'/Title': '3A (Page 1 of 22)', '/Page': IndirectObject(47, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 2 of 22)', '/Page': IndirectObject(51, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 3 of 22)', '/Page': IndirectObject(55, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 4 of 22)', '/Page': IndirectObject(59, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 5 of 22)', '/Page': IndirectObject(63, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 6 of 22)', '/Page': IndirectObject(67, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 7 of 22)', '/Page': IndirectObject(71, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 8 of 22)', '/Page': IndirectObject(75, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 9 of 22)', '/Page': IndirectObject(79, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 10 of 22)', '/Page': IndirectObject(83, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 11 of 22)', '/Page': IndirectObject(88, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 12 of 22)', '/Page': IndirectObject(92, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 13 of 22)', '/Page': IndirectObject(96, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 14 of 22)', '/Page': IndirectObject(100, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 15 of 22)', '/Page': IndirectObject(104, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 16 of 22)', '/Page': IndirectObject(108, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 17 of 22)', '/Page': IndirectObject(112, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 18 of 22)', '/Page': IndirectObject(116, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 19 of 22)', '/Page': IndirectObject(120, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 20 of 22)', '/Page': IndirectObject(124, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 21 of 22)', '/Page': IndirectObject(129, 0), '/Type': '/FitB'}, {'/Title': '3A (Page 22 of 22)', '/Page': IndirectObject(133, 0), '/Type': '/FitB'}], {'/Title': '4A: ALJ Hearing Decision (ALJDEC) Dec. Dt.: 11/12/2015 (24 pages)', '/Page': IndirectObject(137, 0), '/Type': '/FitB'}, [{'/Title': '4A (Page 1 of 24)', '/Page': IndirectObject(137, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 2 of 24)', '/Page': IndirectObject(141, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 3 of 24)', '/Page': IndirectObject(145, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 4 of 24)', '/Page': IndirectObject(149, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 5 of 24)', '/Page': IndirectObject(153, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 6 of 24)', '/Page': IndirectObject(157, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 7 of 24)', '/Page': IndirectObject(161, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 8 of 24)', '/Page': IndirectObject(165, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 9 of 24)', '/Page': IndirectObject(170, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 10 of 24)', '/Page': IndirectObject(174, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 11 of 24)', '/Page': IndirectObject(178, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 12 of 24)', '/Page': IndirectObject(182, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 13 of 24)', '/Page': IndirectObject(186, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 14 of 24)', '/Page': IndirectObject(190, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 15 of 24)', '/Page': IndirectObject(194, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 16 of 24)', '/Page': IndirectObject(198, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 17 of 24)', '/Page': IndirectObject(202, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 18 of 24)', '/Page': IndirectObject(206, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 19 of 24)', '/Page': IndirectObject(211, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 20 of 24)', '/Page': IndirectObject(215, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 21 of 24)', '/Page': IndirectObject(219, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 22 of 24)', '/Page': IndirectObject(223, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 23 of 24)', '/Page': IndirectObject(227, 0), '/Type': '/FitB'}, {'/Title': '4A (Page 24 of 24)', '/Page': IndirectObject(231, 0), '/Type': '/FitB'}]], This for Section F (I skipped the other sections as Section F is what I am particularly interested in and it is the last section in the List {'/Title': 'Section F. Medical Records', '/Page': IndirectObject(838, 0), '/Type': '/FitB'}, [{'/Title': '1F: HIT MER (HITMER) Src.: Kaiser Permanente Tmt. Dt.: 11/21/2014 - 03/17/2016 (85 pages)', '/Page': IndirectObject(838, 0), '/Type': '/FitB'}, [{'/Title': '1F (Page 1 of 85)', '/Page': IndirectObject(838, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 2 of 85)', '/Page': IndirectObject(842, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 3 of 85)', '/Page': IndirectObject(846, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 4 of 85)', '/Page': IndirectObject(850, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 5 of 85)', '/Page': IndirectObject(854, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 6 of 85)', '/Page': IndirectObject(858, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 7 of 85)', '/Page': IndirectObject(862, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 8 of 85)', '/Page': IndirectObject(867, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 9 of 85)', '/Page': IndirectObject(871, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 10 of 85)', '/Page': IndirectObject(875, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 11 of 85)', '/Page': IndirectObject(879, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 12 of 85)', '/Page': IndirectObject(883, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 13 of 85)', '/Page': IndirectObject(887, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 14 of 85)', '/Page': IndirectObject(891, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 15 of 85)', '/Page': IndirectObject(895, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 16 of 85)', '/Page': IndirectObject(899, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 17 of 85)', '/Page': IndirectObject(903, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 18 of 85)', '/Page': IndirectObject(908, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 19 of 85)', '/Page': IndirectObject(912, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 20 of 85)', '/Page': IndirectObject(916, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 21 of 85)', '/Page': IndirectObject(920, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 22 of 85)', '/Page': IndirectObject(924, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 23 of 85)', '/Page': IndirectObject(928, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 24 of 85)', '/Page': IndirectObject(932, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 25 of 85)', '/Page': IndirectObject(936, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 26 of 85)', '/Page': IndirectObject(940, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 27 of 85)', '/Page': IndirectObject(944, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 28 of 85)', '/Page': IndirectObject(949, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 29 of 85)', '/Page': IndirectObject(953, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 30 of 85)', '/Page': IndirectObject(957, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 31 of 85)', '/Page': IndirectObject(961, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 32 of 85)', '/Page': IndirectObject(965, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 33 of 85)', '/Page': IndirectObject(969, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 34 of 85)', '/Page': IndirectObject(973, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 35 of 85)', '/Page': IndirectObject(977, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 36 of 85)', '/Page': IndirectObject(981, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 37 of 85)', '/Page': IndirectObject(985, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 38 of 85)', '/Page': IndirectObject(990, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 39 of 85)', '/Page': IndirectObject(994, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 40 of 85)', '/Page': IndirectObject(998, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 41 of 85)', '/Page': IndirectObject(1002, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 42 of 85)', '/Page': IndirectObject(1006, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 43 of 85)', '/Page': IndirectObject(1010, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 44 of 85)', '/Page': IndirectObject(1014, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 45 of 85)', '/Page': IndirectObject(1018, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 46 of 85)', '/Page': IndirectObject(1022, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 47 of 85)', '/Page': IndirectObject(1026, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 48 of 85)', '/Page': IndirectObject(1031, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 49 of 85)', '/Page': IndirectObject(1035, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 50 of 85)', '/Page': IndirectObject(1039, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 51 of 85)', '/Page': IndirectObject(1043, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 52 of 85)', '/Page': IndirectObject(1047, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 53 of 85)', '/Page': IndirectObject(1051, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 54 of 85)', '/Page': IndirectObject(1055, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 55 of 85)', '/Page': IndirectObject(1059, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 56 of 85)', '/Page': IndirectObject(1063, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 57 of 85)', '/Page': IndirectObject(1067, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 58 of 85)', '/Page': IndirectObject(1072, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 59 of 85)', '/Page': IndirectObject(1076, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 60 of 85)', '/Page': IndirectObject(1080, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 61 of 85)', '/Page': IndirectObject(1084, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 62 of 85)', '/Page': IndirectObject(1088, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 63 of 85)', '/Page': IndirectObject(1092, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 64 of 85)', '/Page': IndirectObject(1096, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 65 of 85)', '/Page': IndirectObject(1100, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 66 of 85)', '/Page': IndirectObject(1104, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 67 of 85)', '/Page': IndirectObject(1108, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 68 of 85)', '/Page': IndirectObject(1113, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 69 of 85)', '/Page': IndirectObject(1117, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 70 of 85)', '/Page': IndirectObject(1121, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 71 of 85)', '/Page': IndirectObject(1125, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 72 of 85)', '/Page': IndirectObject(1129, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 73 of 85)', '/Page': IndirectObject(1133, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 74 of 85)', '/Page': IndirectObject(1137, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 75 of 85)', '/Page': IndirectObject(1141, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 76 of 85)', '/Page': IndirectObject(1145, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 77 of 85)', '/Page': IndirectObject(1149, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 78 of 85)', '/Page': IndirectObject(1154, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 79 of 85)', '/Page': IndirectObject(1158, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 80 of 85)', '/Page': IndirectObject(1162, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 81 of 85)', '/Page': IndirectObject(1166, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 82 of 85)', '/Page': IndirectObject(1170, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 83 of 85)', '/Page': IndirectObject(1174, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 84 of 85)', '/Page': IndirectObject(1178, 0), '/Type': '/FitB'}, {'/Title': '1F (Page 85 of 85)', '/Page': IndirectObject(1182, 0), '/Type': '/FitB'}], {'/Title': '2F: Office Treatment Records (OFFCREC) Src.: MENTAL HEALTH CENTER OF DENVER Tmt. Dt.: 12/22/2015 - 04/14/2016 (76 pages)', '/Page': IndirectObject(1186, 0), '/Type': '/FitB'}, [{'/Title': '2F (Page 1 of 76)', '/Page': IndirectObject(1186, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 2 of 76)', '/Page': IndirectObject(1190, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 3 of 76)', '/Page': IndirectObject(1195, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 4 of 76)', '/Page': IndirectObject(1199, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 5 of 76)', '/Page': IndirectObject(1203, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 6 of 76)', '/Page': IndirectObject(1207, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 7 of 76)', '/Page': IndirectObject(1211, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 8 of 76)', '/Page': IndirectObject(1215, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 9 of 76)', '/Page': IndirectObject(1219, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 10 of 76)', '/Page': IndirectObject(1223, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 11 of 76)', '/Page': IndirectObject(1227, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 12 of 76)', '/Page': IndirectObject(1231, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 13 of 76)', '/Page': IndirectObject(1236, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 14 of 76)', '/Page': IndirectObject(1240, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 15 of 76)', '/Page': IndirectObject(1244, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 16 of 76)', '/Page': IndirectObject(1248, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 17 of 76)', '/Page': IndirectObject(1252, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 18 of 76)', '/Page': IndirectObject(1256, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 19 of 76)', '/Page': IndirectObject(1260, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 20 of 76)', '/Page': IndirectObject(1264, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 21 of 76)', '/Page': IndirectObject(1268, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 22 of 76)', '/Page': IndirectObject(1272, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 23 of 76)', '/Page': IndirectObject(1277, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 24 of 76)', '/Page': IndirectObject(1281, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 25 of 76)', '/Page': IndirectObject(1285, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 26 of 76)', '/Page': IndirectObject(1289, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 27 of 76)', '/Page': IndirectObject(1293, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 28 of 76)', '/Page': IndirectObject(1297, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 29 of 76)', '/Page': IndirectObject(1301, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 30 of 76)', '/Page': IndirectObject(1305, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 31 of 76)', '/Page': IndirectObject(1309, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 32 of 76)', '/Page': IndirectObject(1313, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 33 of 76)', '/Page': IndirectObject(1318, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 34 of 76)', '/Page': IndirectObject(1322, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 35 of 76)', '/Page': IndirectObject(1326, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 36 of 76)', '/Page': IndirectObject(1330, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 37 of 76)', '/Page': IndirectObject(1334, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 38 of 76)', '/Page': IndirectObject(1338, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 39 of 76)', '/Page': IndirectObject(1342, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 40 of 76)', '/Page': IndirectObject(1346, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 41 of 76)', '/Page': IndirectObject(1350, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 42 of 76)', '/Page': IndirectObject(1354, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 43 of 76)', '/Page': IndirectObject(1359, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 44 of 76)', '/Page': IndirectObject(1363, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 45 of 76)', '/Page': IndirectObject(1367, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 46 of 76)', '/Page': IndirectObject(1371, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 47 of 76)', '/Page': IndirectObject(1375, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 48 of 76)', '/Page': IndirectObject(1379, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 49 of 76)', '/Page': IndirectObject(1383, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 50 of 76)', '/Page': IndirectObject(1387, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 51 of 76)', '/Page': IndirectObject(1391, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 52 of 76)', '/Page': IndirectObject(1395, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 53 of 76)', '/Page': IndirectObject(1400, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 54 of 76)', '/Page': IndirectObject(1404, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 55 of 76)', '/Page': IndirectObject(1408, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 56 of 76)', '/Page': IndirectObject(1412, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 57 of 76)', '/Page': IndirectObject(1416, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 58 of 76)', '/Page': IndirectObject(1420, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 59 of 76)', '/Page': IndirectObject(1424, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 60 of 76)', '/Page': IndirectObject(1428, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 61 of 76)', '/Page': IndirectObject(1432, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 62 of 76)', '/Page': IndirectObject(1436, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 63 of 76)', '/Page': IndirectObject(1441, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 64 of 76)', '/Page': IndirectObject(1445, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 65 of 76)', '/Page': IndirectObject(1449, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 66 of 76)', '/Page': IndirectObject(1453, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 67 of 76)', '/Page': IndirectObject(1457, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 68 of 76)', '/Page': IndirectObject(1461, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 69 of 76)', '/Page': IndirectObject(1465, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 70 of 76)', '/Page': IndirectObject(1469, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 71 of 76)', '/Page': IndirectObject(1473, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 72 of 76)', '/Page': IndirectObject(1477, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 73 of 76)', '/Page': IndirectObject(1482, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 74 of 76)', '/Page': IndirectObject(1486, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 75 of 76)', '/Page': IndirectObject(1490, 0), '/Type': '/FitB'}, {'/Title': '2F (Page 76 of 76)', '/Page': IndirectObject(1494, 0), '/Type': '/FitB'}], {'/Title': '3F: Medical Evidence of Record (MER) Src.: Parkland Memorial Hospital Tmt. Dt.: 03/17/2017 - 07/25/2017 (19 pages)', '/Page': IndirectObject(1498, 0), '/Type': '/FitB'}, [{'/Title': '3F (Page 1 of 19)', '/Page': IndirectObject(1498, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 2 of 19)', '/Page': IndirectObject(1502, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 3 of 19)', '/Page': IndirectObject(1506, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 4 of 19)', '/Page': IndirectObject(1510, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 5 of 19)', '/Page': IndirectObject(1514, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 6 of 19)', '/Page': IndirectObject(1518, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 7 of 19)', '/Page': IndirectObject(1523, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 8 of 19)', '/Page': IndirectObject(1527, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 9 of 19)', '/Page': IndirectObject(1531, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 10 of 19)', '/Page': IndirectObject(1535, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 11 of 19)', '/Page': IndirectObject(1539, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 12 of 19)', '/Page': IndirectObject(1543, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 13 of 19)', '/Page': IndirectObject(1547, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 14 of 19)', '/Page': IndirectObject(1551, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 15 of 19)', '/Page': IndirectObject(1555, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 16 of 19)', '/Page': IndirectObject(1559, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 17 of 19)', '/Page': IndirectObject(1564, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 18 of 19)', '/Page': IndirectObject(1568, 0), '/Type': '/FitB'}, {'/Title': '3F (Page 19 of 19)', '/Page': IndirectObject(1572, 0), '/Type': '/FitB'}]]] I am seeking to capture the text at the beginning of each section F exhibit such that I have ordered pairs. In this case Exhibit 1F would be "Kaiser Permanente", the treating source, and "11/21/2014 - 03/17/2016", the date range. Thank you SO MUCH for your help. From larry at hastings.org Sun Feb 4 19:35:03 2018 From: larry at hastings.org (Larry Hastings) Date: Sun, 4 Feb 2018 16:35:03 -0800 Subject: [RELEASED] Python 3.4.8 and Python 3.5.5 are now available Message-ID: <7b3f1a47-bdb1-8601-3a68-2b6c5b53b1ac@hastings.org> On behalf of the Python development community, I'm happy to announce the availability of Python 3.4.8 and Python 3.5.5. Both Python 3.4 and 3.5 are in "security fixes only" mode.? Both versions only accept security fixes, not conventional bug fixes, and both releases are source-only. You can find Python 3.4.8 here: https://www.python.org/downloads/release/python-348/ And you can find Python 3.5.5 here: https://www.python.org/downloads/release/python-355/ Happy Pythoning, //arry/ From qingyun.tao at tophant.com Sun Feb 4 20:23:54 2018 From: qingyun.tao at tophant.com (=?utf-8?B?6Zm26Z2S5LqR?=) Date: Mon, 5 Feb 2018 09:23:54 +0800 Subject: Why not have a recvall method? In-Reply-To: References: Message-ID: Thank you, it's very helpful. I think the recvall should builtin to the _socket module like sendall. ------------------ Original ------------------ From: "Dan Stromberg"; Date: Mon, Feb 5, 2018 06:01 AM To: "???"; Cc: "python-list"; Subject: Re: Why not have a recvall method? On Sun, Feb 4, 2018 at 5:26 AM, ??? wrote: > Hello, all > The socket object has a `sendall` method that can send all bytes you specified. > Oppositely, socket only has a recv method. I wonder why there is not a `recvall` method? > To workaround this, I use `f = socket.makefile('rb')`, then all `f.read(n)` > Thanks. You're probably good with socket.makefile('rb'). An alternative that allows things like reading null-terminated input: http://stromberg.dnsalias.org/~strombrg/bufsock.html I wrote it and have been using it in production for years. From tjreedy at udel.edu Sun Feb 4 20:32:07 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 4 Feb 2018 20:32:07 -0500 Subject: 3.6.4 IDLE In-Reply-To: References: Message-ID: On 2/4/2018 4:09 PM, Rob Alspach wrote: > Hi Everybody! I just installed python 3.6.4 it was a fresh install (no > former versions). I have never used python but have some IS background. The > videos I am seeing online all show the IDE / IDLE; however, it does not > install for me. I am installing from the download > python-3.6.4-embed-amd64.zip. This is for people who want to embed Python in a C program, or something like that. Go back to https://www.python.org/downloads/release/python-364/ and get either a 32-bit x86 or 64-bit x86-64 installer. I suggest the 64-bit if your Win10 is 64-bit, as is likely. I suggest the 'executable' versus the 'web-based' installer, as one can read something else while downloading (less than a minute), and one will have the installer to repair or revise, but that is up to you. -- Terry Jan Reedy From python at mrabarnett.plus.com Sun Feb 4 21:07:19 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 5 Feb 2018 02:07:19 +0000 Subject: 3.6.4 IDLE In-Reply-To: References: Message-ID: <0e6cb20f-1433-cc52-a5b3-97b8be073505@mrabarnett.plus.com> On 2018-02-04 21:09, Rob Alspach wrote: > Hi Everybody! I just installed python 3.6.4 it was a fresh install (no > former versions). I have never used python but have some IS background. The > videos I am seeing online all show the IDE / IDLE; however, it does not > install for me. I am installing from the download > python-3.6.4-embed-amd64.zip. I am running windows 10. I also have Oracle > on this laptop. > > The file does not have an installer and also does not write to the file > path. I have already uninstalled and reinstalled the problem remains. I can > update the path no problem but where can I get the IDLE / IDE to follow > along with the videos on youtube? I am using notepad currently but have no > way to execute any files I write. If I save a file in notepad and open > through python the python window terminates when the operation is complete > (much to fast for these old eyes). > python-3.6.4-embed-amd64.zip is for "embedded, small or minimal hardware devices". I think you probably don't want that one! I always use the "executable installer". From dallasdisabilityattorney at gmail.com Sun Feb 4 23:06:27 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Sun, 4 Feb 2018 20:06:27 -0800 (PST) Subject: Parsing Nested List In-Reply-To: References: <489f288d-11b9-4b28-94d6-2ac7c14f49b6@googlegroups.com> Message-ID: On Sunday, February 4, 2018 at 5:32:51 PM UTC-6, Stanley Denman wrote: > On Sunday, February 4, 2018 at 4:26:24 PM UTC-6, Stanley Denman wrote: > > I am trying to parse a Python nested list that is the result of the getOutlines() function of module PyPFD2 using pyparsing module. This is the result I get. what in the world are 'expandtabs' and why is that making a difference to my parse attempt? > > > > Python Code > > 7 > > import PPDF2,pyparsing > > from pyparsing import Word, alphas, nums > > pdfFileObj=open('x.pdf','rb') > > pdfReader=PyPDF2.PdfFileReader(pdfFileObj) > > List=pdfReader.getOutlines() > > myparser = Word( alphas ) + Word(nums, exact=2) +"of" + Word(nums, exact=2) > > myparser.parseString(List) > > > > This is the error I get: > > > > Traceback (most recent call last): > > File "", line 1, in > > myparser.parseString(List) > > File "C:\python\lib\site-packages\pyparsing.py", line 1620, in parseString > > instring = instring.expandtabs() > > AttributeError: 'list' object has no attribute 'expandtabs' > > > > Thanks so much, not getting any helpful responses from https://python-forum.io. I have found that I can use the index values in the list to print out the section I need. So print(MyList[7]) get me to section f taht I want. print(MyList[9][1]) for example give me a string that is the bookmark entry for Exhibit 1F. But this index value would presumeably be different for each pdf file - that is there may not always be Section A-E, but there will always be a Section F. In ther words, the index values that get me to the right section would be different in each pdf file. From dieter at handshake.de Mon Feb 5 02:07:02 2018 From: dieter at handshake.de (dieter) Date: Mon, 05 Feb 2018 08:07:02 +0100 Subject: auto-correct a speech-to-text output and relate to of the words based on syllables References: <78b57c55-b57f-4e80-b390-5fb56df4012b@googlegroups.com> <87o9l7983o.fsf@handshake.de> <87372icvym.fsf@handshake.de> <20180203120940.6jqtr6s7qr7mlhkx@hjp.at> Message-ID: <87372fhq3t.fsf@handshake.de> "Peter J. Holzer" writes: > On 2018-02-03 09:34:57 +0100, dieter wrote: > ... > The difficulty is to *recognise* it correctly. Was that tangle of sound > waves an "l" or an "r"? This not as unambiguous as you seem to think. > So a speech-to-text program may hear "right" when the speaker was really > saying "light". If you have only the output from that program you must > determine whether "right" is correct or must be corrected to "light". The primary recommendation from my first response has been: determine the step in the process chain that made the error. If already the first step ("speech-to-text") has wrongly interpreted "light" as "right", then obviously context is necessary to determine that this was wrong. From frank at chagford.com Mon Feb 5 02:35:35 2018 From: frank at chagford.com (Frank Millman) Date: Mon, 5 Feb 2018 09:35:35 +0200 Subject: Why no '|' operator for dict? Message-ID: Hi all I recently learned that you can create a set 'on-the-fly' from two existing sets using the '|' operator - Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> >>> set_1 = set(('a', 'b', 'c')) >>> set_2 = set(('d',)) >>> set_1 | set_2 {'d', 'a', 'c', 'b'} >>> I was hoping that I could do the same with a dictionary, but it does not work - >>> dict_1 = {1: 'one', 2: 'two'} >>> dict_2 = {3: 'three'} >>> dict_1 | dict_2 Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for |: 'dict' and 'dict' >>> The best that I can come up with is - >>> dict([(k, v) for k, v in dict_1.items()] + [(k, v) for k, v in >>> dict_2.items()]) {1: 'one', 2: 'two', 3: 'three'} >>> So I have 2 questions - 1. Is there any particular reason why '|' is not supported? 2. Is there a better way to do what I want? Thanks Frank Millman From dieter at handshake.de Mon Feb 5 02:37:23 2018 From: dieter at handshake.de (dieter) Date: Mon, 05 Feb 2018 08:37:23 +0100 Subject: Help on PyImport_Import(pNAme) References: Message-ID: <87y3k7ga4s.fsf@handshake.de> Jason Qian via Python-list writes: > This only works when loading modules from the current directory. > Is there a way I can load from somewhere else ? Have a look at the module `importlib` and especially its function `import_module`. Python knows about two forms of "import": "absolute import" and "relative import". An "absolute import" tries (in the usual case) to import from a list of directories, maintained in "sys.path"; a "relative import" (recognized via the module name starting with a .") relative to a package. Section 31.5.6.3 of "https://docs.python.org/3/library/importlib.html#module-importlib.util" tells you how to import a source file directly (independent from any context). From ian.g.kelly at gmail.com Mon Feb 5 03:14:53 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 5 Feb 2018 01:14:53 -0700 Subject: Why no '|' operator for dict? In-Reply-To: References: Message-ID: On Mon, Feb 5, 2018 at 12:35 AM, Frank Millman wrote: > So I have 2 questions - > > 1. Is there any particular reason why '|' is not supported? '|' is the set union operation, roughly equivalent to the set.union method. Dicts don't have a union operation. If they did, and the same key were found in both sets, what would be the value of that key in the union? > 2. Is there a better way to do what I want? The dict.items() view is explicitly set-like and can be unioned, so you can do this: py> dict(d1.items() | d2.items()) As to the question of which value will appear in the union in the case of duplicate keys, it will be whichever one arbitrarily appears later in the iteration order of the intermediate set. From frank at chagford.com Mon Feb 5 03:31:10 2018 From: frank at chagford.com (Frank Millman) Date: Mon, 5 Feb 2018 10:31:10 +0200 Subject: Why no '|' operator for dict? In-Reply-To: References: Message-ID: "Ian Kelly" wrote in message news:CALwzidkP3Ls4s-Zi3ax6no-68kw4_XDOzvWA-cJ+oz+ApqrWXg at mail.gmail.com... > > On Mon, Feb 5, 2018 at 12:35 AM, Frank Millman wrote: > > So I have 2 questions - > > > > 1. Is there any particular reason why '|' is not supported? > > '|' is the set union operation, roughly equivalent to the set.union > method. Dicts don't have a union operation. If they did, and the same > key were found in both sets, what would be the value of that key in > the union? > > > 2. Is there a better way to do what I want? > > The dict.items() view is explicitly set-like and can be unioned, so > you can do this: > > py> dict(d1.items() | d2.items()) > Excellent explanation, and excellent solution! Thanks very much. Frank From maxischmeii at gmail.com Mon Feb 5 04:29:42 2018 From: maxischmeii at gmail.com (Maxime S) Date: Mon, 5 Feb 2018 10:29:42 +0100 Subject: Why no '|' operator for dict? In-Reply-To: References: Message-ID: 2018-02-05 9:14 GMT+01:00 Ian Kelly : > On Mon, Feb 5, 2018 at 12:35 AM, Frank Millman wrote: >> 2. Is there a better way to do what I want? > > The dict.items() view is explicitly set-like and can be unioned, so > you can do this: > > py> dict(d1.items() | d2.items()) > > As to the question of which value will appear in the union in the case > of duplicate keys, it will be whichever one arbitrarily appears later > in the iteration order of the intermediate set. Since Python 3.5, it is also possible to use PEP448 generalized unpacking: dict([*d1.items(), *d2.items()]) In which case the value that appears in case of duplicate keys is better defined, it will be the one appearing in the last dictionnary. From tjreedy at udel.edu Mon Feb 5 04:34:00 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 5 Feb 2018 04:34:00 -0500 Subject: Why no '|' operator for dict? In-Reply-To: References: Message-ID: On 2/5/2018 2:35 AM, Frank Millman wrote: > I recently learned that you can create a set 'on-the-fly' from two > existing sets using the '|' operator - > > Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 > bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> >>>> set_1 = set(('a', 'b', 'c')) >>>> set_2 = set(('d',)) >>>> set_1 | set_2 > {'d', 'a', 'c', 'b'} > I was hoping that I could do the same with a dictionary, but it does not > work - If you know keys are unique or you want keep the second value for a duplicate (equal) key, >>> d1 = {1: 'one', 2: 'two'}; d2 = {3: 'three', 1:'One'} >>> d3 = d1.copy() >>> d3.update(d2) >>> d3 {1: 'One', 2: 'two', 3: 'three'} -- Terry Jan Reedy From storchaka at gmail.com Mon Feb 5 04:37:30 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 5 Feb 2018 11:37:30 +0200 Subject: Why no '|' operator for dict? In-Reply-To: References: Message-ID: 05.02.18 10:14, Ian Kelly ????: > On Mon, Feb 5, 2018 at 12:35 AM, Frank Millman wrote: >> So I have 2 questions - >> >> 1. Is there any particular reason why '|' is not supported? > > '|' is the set union operation, roughly equivalent to the set.union > method. Dicts don't have a union operation. If they did, and the same > key were found in both sets, what would be the value of that key in > the union? > >> 2. Is there a better way to do what I want? > > The dict.items() view is explicitly set-like and can be unioned, so > you can do this: > > py> dict(d1.items() | d2.items()) This doesn't work with non-hashable values. The simplest (and perhaps the most efficient) way in recent Python versions is: {**d1, **d2} In old Python versions this should be written as d = dict(d1) d.update(d2) From steve+comp.lang.python at pearwood.info Mon Feb 5 04:59:55 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 5 Feb 2018 09:59:55 +0000 (UTC) Subject: Why no '|' operator for dict? References: Message-ID: On Mon, 05 Feb 2018 01:14:53 -0700, Ian Kelly wrote: > On Mon, Feb 5, 2018 at 12:35 AM, Frank Millman > wrote: >> So I have 2 questions - >> >> 1. Is there any particular reason why '|' is not supported? > > '|' is the set union operation, roughly equivalent to the set.union > method. Dicts don't have a union operation. If they did, and the same > key were found in both sets, what would be the value of that key in the > union? Obviously it should be a quantum superposition of the two values, which remains uncertain until such time as you actually print the value and observe it. -- Steve From gbs.deadeye at gmail.com Mon Feb 5 08:17:59 2018 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Mon, 05 Feb 2018 13:17:59 +0000 Subject: Why no '|' operator for dict? In-Reply-To: References: Message-ID: You can use keyword-argument unpacking in a dict-constructor. Values of duplicate keys are overwritten from left to right. The last wins. >>> dict1 = {'foo': 13, 'bar': 42} >>> dict2 = {'foo': 42, 'hello': 'world'} >>> {**dict1, **dict2} {'bar': 42, 'foo': 42, 'hello': 'world'} {**dict2, **dict1} {'bar': 42, 'foo': 13, 'hello': 'world'} You can make a Class for this task, if you need it very often: class UDict(dict): def __or__(self, other): if not isinstance(other, (self.__class__, dict)): raise ValueError('Is not a dict!') return {**self, **other} __ror__ = __or__ >>> UDict({'foo': 1, 'bar': 1337}) | UDict({'bar': 43}) {'bar': 43, 'foo': 1} >>> UDict({'foo': 1, 'bar': 1337}) | {'bar': 43} {'bar': 43, 'foo': 1} >>> {'foo':42} | UDict({'foo': 1, 'bar': 1337}) {'bar': 1337, 'foo': 42} Greetings Andre Steven D'Aprano schrieb am Mo., 5. Feb. 2018 um 11:03 Uhr: > On Mon, 05 Feb 2018 01:14:53 -0700, Ian Kelly wrote: > > > On Mon, Feb 5, 2018 at 12:35 AM, Frank Millman > > wrote: > >> So I have 2 questions - > >> > >> 1. Is there any particular reason why '|' is not supported? > > > > '|' is the set union operation, roughly equivalent to the set.union > > method. Dicts don't have a union operation. If they did, and the same > > key were found in both sets, what would be the value of that key in the > > union? > > Obviously it should be a quantum superposition of the two values, which > remains uncertain until such time as you actually print the value and > observe it. > > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From Cecil at decebal.nl Mon Feb 5 08:25:46 2018 From: Cecil at decebal.nl (Cecil Westerhof) Date: Mon, 05 Feb 2018 14:25:46 +0100 Subject: From recovery.js to recoveryjsonlz4 Message-ID: <87372fpnz9.fsf@munus.decebal.nl> I have a script to get the number of windows and tabs that firefox uses. It always used a file recovery.js, but it changed to recovery.jsonlz4. Looking at the extension I would think it is an lz4 compressed file. But when I use: import lz4 I see that it is deprecated. How should I work with this file? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From preductor at gmail.com Mon Feb 5 09:53:33 2018 From: preductor at gmail.com (=?UTF-8?B?0JTQtdC90LjRgSDQntC70LXQs9C+0LLQuNGH?=) Date: Mon, 5 Feb 2018 17:53:33 +0300 Subject: Fwd: Problem during setup In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: ????? ???????? Date: Sun, Feb 4, 2018 at 12:54 PM Subject: Problem during setup To: python-list at python.org I tried to install python 3.5 and python 3.6, but the same mistake interrupt process "Windows 7 Service Pack 1 applicable updates are required" Log file attached. I tried to install some updates for Windows, but unsuccessully, may be I don' t know which one I need. And what strange - python 3.4 installs and works well. Can you advise something? -------------- next part -------------- [0A00:0D20][2018-02-04T12:47:55]i001: Burn v3.10.3.3007, Windows v6.1 (Build 7600: Service Pack 0), path: C:\Users\1249~1\AppData\Local\Temp\{EDC3FDFD-6DD4-41E4-90E0-322D93C60ACA}\.cr\python-3.6.4-amd64.exe [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'ActionLikeInstalling' to value 'Installing' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'ActionLikeInstallation' to value 'Setup' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'ShortVersion' to value '3.6' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'ShortVersionNoDot' to value '36' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'WinVer' to value '3.6' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'WinVerNoDot' to value '36' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'InstallAllUsers' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'InstallLauncherAllUsers' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'TargetDir' to value '' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'DefaultAllUsersTargetDir' to value '[ProgramFiles64Folder]Python[WinVerNoDot]' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'TargetPlatform' to value 'x64' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'DefaultJustForMeTargetDir' to value '[LocalAppDataFolder]Programs\Python\Python[WinVerNoDot]' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'OptionalFeaturesRegistryKey' to value 'Software\Python\PythonCore\[WinVer]\InstalledFeatures' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'TargetDirRegistryKey' to value 'Software\Python\PythonCore\[WinVer]\InstallPath' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'DefaultCustomTargetDir' to value '' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'InstallAllUsersState' to value 'enabled' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'InstallLauncherAllUsersState' to value 'enabled' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'CustomInstallLauncherAllUsersState' to value '[InstallLauncherAllUsersState]' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'TargetDirState' to value 'enabled' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'CustomBrowseButtonState' to value 'enabled' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_core' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_exe' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_dev' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_lib' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_test' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_doc' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_tools' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_tcltk' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_pip' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_launcher' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'Include_launcherState' to value 'enabled' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_symbols' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Include_debug' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'LauncherOnly' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'DetectedLauncher' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'DetectedOldLauncher' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'AssociateFiles' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'Shortcuts' to value '1' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'PrependPath' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'CompileAll' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing numeric variable 'SimpleInstall' to value '0' [0A00:0D20][2018-02-04T12:47:55]i000: Initializing string variable 'SimpleInstallDescription' to value '' [0A00:0D20][2018-02-04T12:47:55]i009: Command Line: '"-burn.clean.room=C:\Users\????? ?????????\Downloads\python-3.6.4-amd64.exe" -burn.filehandle.attached=176 -burn.filehandle.self=184' [0A00:0D20][2018-02-04T12:47:55]i000: Setting string variable 'WixBundleOriginalSource' to value 'C:\Users\????? ?????????\Downloads\python-3.6.4-amd64.exe' [0A00:0D20][2018-02-04T12:47:55]i000: Setting string variable 'WixBundleOriginalSourceFolder' to value 'C:\Users\????? ?????????\Downloads\' [0A00:0D20][2018-02-04T12:47:55]i000: Setting string variable 'WixBundleLog' to value 'C:\Users\1249~1\AppData\Local\Temp\Python 3.6.4 (64-bit)_20180204124755.log' [0A00:0D20][2018-02-04T12:47:55]i000: Setting string variable 'WixBundleName' to value 'Python 3.6.4 (64-bit)' [0A00:0D20][2018-02-04T12:47:55]i000: Setting string variable 'WixBundleManufacturer' to value 'Python Software Foundation' [0A00:0D20][2018-02-04T12:47:55]i000: Setting numeric variable 'CRTInstalled' to value 1 [0A00:0E20][2018-02-04T12:47:57]i000: Setting string variable 'ActionLikeInstalling' to value 'Installing' [0A00:0E20][2018-02-04T12:47:57]i000: Setting string variable 'ActionLikeInstallation' to value 'Setup' [0A00:0E20][2018-02-04T12:47:57]i000: Setting version variable 'WixBundleFileVersion' to value '3.6.4150.0' [0A00:0E20][2018-02-04T12:47:57]e000: Detected Windows 7 RTM [0A00:0E20][2018-02-04T12:47:57]e000: Service Pack 1 is required to continue installation From torriem at gmail.com Mon Feb 5 12:10:31 2018 From: torriem at gmail.com (Michael Torrie) Date: Mon, 5 Feb 2018 10:10:31 -0700 Subject: Fwd: Problem during setup In-Reply-To: References: Message-ID: <40ab7e7f-5e41-fcfc-63c9-863eea294762@gmail.com> On 02/05/2018 07:53 AM, ????? ???????? wrote: > I tried to install python 3.5 and python 3.6, but the same mistake > interrupt process > "Windows 7 Service Pack 1 applicable updates are required" Log file > attached. I tried to install some updates for Windows, but unsuccessully, > may be I don' t know which one I need. And what strange - python 3.4 > installs and works well. Can you advise something? Is there a reason you're not running Service Pack 1? Is this computer isolated from the internet? Python 3.5 installer requires that you're at least mostly up to date on your windows updates. There's no easy way around that. If you're unable to use Windows Update to bring things up to date, there's not much we can recommend, other than to move to a free operating system such as Linux. From darkorbitaknaentou at centrum.cz Mon Feb 5 12:13:26 2018 From: darkorbitaknaentou at centrum.cz (darkorbitaknaentou at centrum.cz) Date: Mon, 05 Feb 2018 18:13:26 +0100 Subject: =?utf-8?q?Python_=22Bad_syntax=22?= Message-ID: <20180205181326.C3D77EAB@centrum.cz> ? Hi, I have a problem in continuing the function. ? I'm a beginner, I'm learning from a textbook. I'm going to put the following examples from a textbook that displays "wrong syntax" ? >>> for letter in "ABCDEFGHIJKLMNOPQRSTUVWXYZ": ? ? ? ? ? ? if letter in "AEIOU": ? ? ? ? ? ? ? ? print(letter, "is a vowel") ? ? ? ? ? ?else: ? ? ? ? ? ? ? ? ?print(letter, "is a consonant") ? In this text, I will write a "wrong syntax" after confirming the "else" function. How is it possible? Using the Bad Version of Python? Please, please, thank you very much! ? The same error is in the "break" function: ? >>> while True: ? ? ? ? ? ? ? item = get_next_item() ? ? ? ? ? ? ? if not item: ? ? ? ? ? ? ? ? ? ?break ? ? ? ? ? ? ? process_item(item) ? Here 'wrong syntax' occurs after the "break". ? How is it possible? Bad version of Python? Which version to use? ? Thank you for any advice and help! ? --? https://mail.python.org/mailman/listinfo/python-list ? ? ? ? From ian.g.kelly at gmail.com Mon Feb 5 12:25:04 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 5 Feb 2018 10:25:04 -0700 Subject: Python "Bad syntax" In-Reply-To: <20180205181326.C3D77EAB@centrum.cz> References: <20180205181326.C3D77EAB@centrum.cz> Message-ID: On Mon, Feb 5, 2018 at 10:13 AM, wrote: > > Hi, I have a problem in continuing the function. > > I'm a beginner, I'm learning from a textbook. I'm going to put the following > examples from a textbook that displays "wrong syntax" It would be very helpful if you would copy/paste the exact error message instead of just paraphasing it. > >>>> >>>> for letter in "ABCDEFGHIJKLMNOPQRSTUVWXYZ": > > if letter in "AEIOU": > print(letter, "is a vowel") > else: > print(letter, "is a consonant") > > In this text, I will write a "wrong syntax" after confirming the "else" > function. How is it possible? Using the Bad Version of Python? Please, > please, thank you very much! Copying this into either Python 2.7 or 3.5, I get IndentationError (because the indentation of the if and the else is inconsistent), not SyntaxError. I don't know whether that's what you're seeing or if it was just a copying error when you wrote the email. > The same error is in the "break" function: > >>>> >>>> while True: > > item = get_next_item() > if not item: > break > process_item(item) > > Here 'wrong syntax' occurs after the "break". I don't get a syntax error from this at all. I just get NameError because get_next_item is not defined. If it and process_item were both defined, I expect this would run. From philb at philb.ca Mon Feb 5 12:28:56 2018 From: philb at philb.ca (Phil Boutros) Date: Mon, 5 Feb 2018 17:28:56 -0000 (UTC) Subject: Python "Bad syntax" References: <20180205181326.C3D77EAB@centrum.cz> Message-ID: wrote: > ? > Hi, I have a problem in continuing the function. > ? > I'm a beginner, I'm learning from a textbook. I'm going to put the > following examples from a textbook that displays "wrong syntax" > ? >>>> for letter in "ABCDEFGHIJKLMNOPQRSTUVWXYZ": > ? ? ? ? ? ? if letter in "AEIOU": > ? ? ? ? ? ? ? ? print(letter, "is a vowel") > ? ? ? ? ? ?else: > ? ? ? ? ? ? ? ? ?print(letter, "is a consonant") > ? > In this text, I will write a "wrong syntax" after confirming the > "else" function. How is it possible? Using the Bad Version of > Python? Please, please, thank you very much! Which version of python are you using? That syntax for "print" started in python 3 (since print became a function). Try adding: from __future__ import print_function before your code if you're still using python 2.x Phil From auriocus at gmx.de Mon Feb 5 12:30:01 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Mon, 5 Feb 2018 18:30:01 +0100 Subject: Python "Bad syntax" In-Reply-To: References: <20180205181326.C3D77EAB@centrum.cz> Message-ID: Am 05.02.18 um 18:13 schrieb darkorbitaknaentou at centrum.cz: > > Hi, I have a problem in continuing the function. > > I'm a beginner, I'm learning from a textbook. I'm going to put the > following examples from a textbook that displays "wrong syntax" > >>>> for letter in "ABCDEFGHIJKLMNOPQRSTUVWXYZ": > ? ? ? ? ? ? if letter in "AEIOU": > ? ? ? ? ? ? ? ? print(letter, "is a vowel") > ? ? ? ? ? ?else: > ? ? ? ? ? ? ? ? ?print(letter, "is a consonant") > > In this text, I will write a "wrong syntax" after confirming the "else" > function. How is it possible? Using the Bad Version of Python? Please, > please, thank you very much! Make sure you don't mix tabs and spaces and make consistent indentation. In your above example, the problem is that "if" is indented one more space than "else". If I paste this into ipython (3.6.1), I get: File "", line 4 else: ^ IndentationError: unindent does not match any outer indentation level which is a pretty clear error. I'm not sure which version of Python you are using which only displays "wrong syntax" instead of a descriptive error message. I suggest in any case to use IPython for interactive use. > > The same error is in the "break" function: > >>>> while True: > ? ? ? ? ? ? ? item = get_next_item() > ? ? ? ? ? ? ? if not item: > ? ? ? ? ? ? ? ? ? ?break > ? ? ? ? ? ? ? process_item(item) I can't see an error here, it only fails for me because the functions "get_next_item" and "process_item" are not defined. Maybe the tabs/spaces were mixed in your original input? Christian From philb at philb.ca Mon Feb 5 12:32:25 2018 From: philb at philb.ca (Phil Boutros) Date: Mon, 5 Feb 2018 17:32:25 -0000 (UTC) Subject: Python "Bad syntax" References: <20180205181326.C3D77EAB@centrum.cz> Message-ID: Phil Boutros wrote: > > Which version of python are you using? That syntax for "print" > started in python 3 (since print became a function). > > Try adding: > > from __future__ import print_function > > before your code if you're still using python 2.x Altough, testing it in Python2, it still works, but it prints the tuples, of course. As Christian said, I suspect this is an indent error of sorts. Manually typing your code with correct indents, I can verify that it works in both Python 2 and 3 (with various results depending on which version of print is used, of course.) Phil -- AH#61 Wolf#14 BS#89 bus#1 CCB#1 SENS KOTC#4 philb at philb.ca http://philb.ca From bob at mellowood.ca Mon Feb 5 12:33:35 2018 From: bob at mellowood.ca (Bob van der Poel) Date: Mon, 5 Feb 2018 10:33:35 -0700 Subject: Python "Bad syntax" In-Reply-To: <20180205181326.C3D77EAB@centrum.cz> References: <20180205181326.C3D77EAB@centrum.cz> Message-ID: On Mon, Feb 5, 2018 at 10:13 AM, wrote: > > Hi, I have a problem in continuing the function. > > I'm a beginner, I'm learning from a textbook. I'm going to put the > following examples from a textbook that displays "wrong syntax" > > >> for letter in "ABCDEFGHIJKLMNOPQRSTUVWXYZ": >>>> >>> if letter in "AEIOU": > print(letter, "is a vowel") > else: > print(letter, "is a consonant") > > In this text, I will write a "wrong syntax" after confirming the "else" > function. How is it possible? Using the Bad Version of Python? Please, > please, thank you very much! > > The same error is in the "break" function: > > >> while True: >>>> >>> item = get_next_item() > if not item: > break > process_item(item) > > Here 'wrong syntax' occurs after the "break". > > How is it possible? Bad version of Python? Which version to use? > > Thank you for any advice and help! > > > Check that the code is properly indented. I did a cut/paste and then manually indented the first example and it works fine. -- **** Listen to my FREE CD at http://www.mellowood.ca/music/cedars **** Bob van der Poel ** Wynndel, British Columbia, CANADA ** EMAIL: bob at mellowood.ca WWW: http://www.mellowood.ca From rosuav at gmail.com Mon Feb 5 12:33:41 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 6 Feb 2018 04:33:41 +1100 Subject: Python "Bad syntax" In-Reply-To: References: <20180205181326.C3D77EAB@centrum.cz> Message-ID: On Tue, Feb 6, 2018 at 4:28 AM, Phil Boutros wrote: > wrote: >> >> Hi, I have a problem in continuing the function. >> >> I'm a beginner, I'm learning from a textbook. I'm going to put the >> following examples from a textbook that displays "wrong syntax" >> >>>>> for letter in "ABCDEFGHIJKLMNOPQRSTUVWXYZ": >> if letter in "AEIOU": >> print(letter, "is a vowel") >> else: >> print(letter, "is a consonant") >> >> In this text, I will write a "wrong syntax" after confirming the >> "else" function. How is it possible? Using the Bad Version of >> Python? Please, please, thank you very much! > > Which version of python are you using? That syntax for "print" > started in python 3 (since print became a function). > > Try adding: > > from __future__ import print_function > > before your code if you're still using python 2.x > It would still be valid syntax in Python 2 - it'd just print out a tuple. ChrisA From breamoreboy at gmail.com Mon Feb 5 13:20:26 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 5 Feb 2018 10:20:26 -0800 (PST) Subject: From recovery.js to recoveryjsonlz4 In-Reply-To: <87372fpnz9.fsf@munus.decebal.nl> References: <87372fpnz9.fsf@munus.decebal.nl> Message-ID: On Monday, February 5, 2018 at 1:28:16 PM UTC, Cecil Westerhof wrote: > I have a script to get the number of windows and tabs that firefox > uses. It always used a file recovery.js, but it changed to > recovery.jsonlz4. > > Looking at the extension I would think it is an lz4 compressed file. > But when I use: > import lz4 > > I see that it is deprecated. How should I work with this file? > > -- > Cecil Westerhof > Senior Software Engineer > LinkedIn: http://www.linkedin.com/in/cecilwesterhof I can't see anything here http://python-lz4.readthedocs.io/en/stable/lz4.html to indicate that the package is deprecated, although the lz4version function has been since version 0.14. Further the latest 0.21.6 was released just yesterday, 04/02/2018, so I'm guessing that you've just misread something. -- Kindest regards. Mark Lawrence. From phd at phdru.name Mon Feb 5 13:35:15 2018 From: phd at phdru.name (Oleg Broytman) Date: Mon, 5 Feb 2018 19:35:15 +0100 Subject: Django-CheetahTemplate 0.2 Message-ID: <20180205183515.GA12385@phdru.name> Hello! Django-CheetahTemplate version 0.2. WHAT IS Django-CheetahTemplate Django-CheetahTemplate is a Django template backend to use CheetahTemplate3 in Django. It's a brand new project created for the new custom Django template backends API. It works with Python 2.7 or Python 3.4+, Django 1.11 and 2+, CheetahTemplate3. WHAT'S NEW Version 0.2.0 (2018-02-05) The first public release. WHERE TO GET Home Page: https://github.com/CheetahTemplate3/django-cheetahtemplate PyPI: https://pypi.python.org/pypi/django-cheetahtemplate AUTHOR Oleg Broytman COPYRIGHT Copyright (C) 2018 PhiloSoft Design. LICENSE MIT Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From alain at universite-de-strasbourg.fr.invalid Mon Feb 5 14:47:44 2018 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Mon, 05 Feb 2018 20:47:44 +0100 Subject: "None" and "pass" References: Message-ID: <87vafbw74v.fsf@universite-de-strasbourg.fr.invalid> ram at zedat.fu-berlin.de (Stefan Ram) writes: > A participant of my Python course asked whether one could > also use "None" instead of "pass". What do you think? > > def f(): > pass > > can also be written as > > def f(): > None > > . Is there any place where "None" could not be used > instead of "pass"? No, an expression is always a valid statement: https://docs.python.org/3.6/reference/simple_stmts.html Use None, or 42+0, or 1+1+1+1+1+1+1+1+1, or whatever you want (that does not have any side-effect and/or throw an exception). And be fired right after your first code review. -- Alain. From rosuav at gmail.com Mon Feb 5 15:48:07 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 6 Feb 2018 07:48:07 +1100 Subject: "None" and "pass" In-Reply-To: <87vafbw74v.fsf@universite-de-strasbourg.fr.invalid> References: <87vafbw74v.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Tue, Feb 6, 2018 at 6:47 AM, Alain Ketterlin wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes: > >> A participant of my Python course asked whether one could >> also use "None" instead of "pass". What do you think? >> >> def f(): >> pass >> >> can also be written as >> >> def f(): >> None >> >> . Is there any place where "None" could not be used >> instead of "pass"? > > No, an expression is always a valid statement: > > https://docs.python.org/3.6/reference/simple_stmts.html > > Use None, or 42+0, or 1+1+1+1+1+1+1+1+1, or whatever you want (that does > not have any side-effect and/or throw an exception). And be fired right > after your first code review. > As one special case, I would accept this sort of code: def f(): ... (three dots representing the special value Ellipsis) It's a great short-hand for "stub". Otherwise, though, an expression that isn't used (like putting "1+1" on a line on its own) should normally fail a code review. ChrisA From hjp-python at hjp.at Mon Feb 5 16:07:13 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Mon, 5 Feb 2018 22:07:13 +0100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: <20180205210713.sbexpuwy6a5gbvnn@hjp.at> On 2018-01-28 15:04:26 +0000, Steven D'Aprano wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. For your amusment, here is how a well-known German tech news site publishes source-code: https://video.golem.de/files/1/9/20542/twitterreplybotpython.pdf?start=0.00 hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ben+python at benfinney.id.au Mon Feb 5 16:39:39 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 06 Feb 2018 08:39:39 +1100 Subject: "None" and "pass" References: <87vafbw74v.fsf@universite-de-strasbourg.fr.invalid> Message-ID: <85shafglpg.fsf@benfinney.id.au> Chris Angelico writes: > As one special case, I would accept this sort of code: > > def f(): > ... > > (three dots representing the special value Ellipsis) > > It's a great short-hand for "stub". I would not accept that. An even better way to write a stub function is to write a docstring: def frobnicate(): """ Frobnicate the spangule. """ A docstring, like any bare expression, is also a valid statement. Writing a docstring can be done immediately, because if you're writing a stub function you at least know the external interface of that function. -- \ ?If you do not trust the source do not use this program.? | `\ ?Microsoft Vista security dialogue | _o__) | Ben Finney From rosuav at gmail.com Mon Feb 5 16:55:35 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 6 Feb 2018 08:55:35 +1100 Subject: "None" and "pass" In-Reply-To: <85shafglpg.fsf@benfinney.id.au> References: <87vafbw74v.fsf@universite-de-strasbourg.fr.invalid> <85shafglpg.fsf@benfinney.id.au> Message-ID: On Tue, Feb 6, 2018 at 8:39 AM, Ben Finney wrote: > Chris Angelico writes: > >> As one special case, I would accept this sort of code: >> >> def f(): >> ... >> >> (three dots representing the special value Ellipsis) >> >> It's a great short-hand for "stub". > > I would not accept that. > > An even better way to write a stub function is to write a docstring: > > def frobnicate(): > """ Frobnicate the spangule. """ > > A docstring, like any bare expression, is also a valid statement. > Writing a docstring can be done immediately, because if you're writing a > stub function you at least know the external interface of that function. > This is true, but I'd rather have something _under_ the docstring if possible, and "..." works well for that. A docstring with nothing underneath doesn't look like a stub - it looks like a failed edit or something. Having a placeholder shows that it's intentional. ChrisA From matt.priest at intel.com Mon Feb 5 18:11:02 2018 From: matt.priest at intel.com (Priest, Matt) Date: Mon, 5 Feb 2018 23:11:02 +0000 Subject: libxml2 installation/binding issue with Python 3.6.4 Message-ID: <68B69E4C07B04A488BA8528DF70CA165844150F7@fmsmsx116.amr.corp.intel.com> Hello, I am not sure if this is the correct place to post an issue/question like this, but here goes... I've successfully (?) installed Python 3.6.4 and libxml2, with the ultimate goal of installing GTK+ 3.22.0. However, I'm running into this error: python3 Python 3.6.4 (default, Feb 5 2018, 13:28:04) [GCC 4.7.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import libxml2 Traceback (most recent call last): File "", line 1, in File "/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2.py", line 1, in import libxml2mod ImportError: /nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2mod.so: undefined symbol: _PyVerify_fd Here are the details on the version, cflags, and ldflags. python3 --version ; Python 3.6.4 python3-config --cflags -I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/include/python3.6m -I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/include/python3.6m -Wno-unused-result -Wsign-compare -fPIC -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes python3-config -ldflags; -L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/config-3.6m-x86_64-linux-gnu -L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib -lpython3.6m -lpthread -ldl -lutil -lrt -lm -Xlinker -export-dynamic Anyhelp or hint would be appreciated... Matt From tjreedy at udel.edu Mon Feb 5 21:37:37 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 5 Feb 2018 21:37:37 -0500 Subject: libxml2 installation/binding issue with Python 3.6.4 In-Reply-To: <68B69E4C07B04A488BA8528DF70CA165844150F7@fmsmsx116.amr.corp.intel.com> References: <68B69E4C07B04A488BA8528DF70CA165844150F7@fmsmsx116.amr.corp.intel.com> Message-ID: On 2/5/2018 6:11 PM, Priest, Matt wrote: > Hello, > > I am not sure if this is the correct place to post an issue/question like this, but here goes... > > I've successfully (?) installed Python 3.6.4 and libxml2, with the ultimate goal of installing GTK+ 3.22.0. > However, I'm running into this error: > > > python3 > Python 3.6.4 (default, Feb 5 2018, 13:28:04) > [GCC 4.7.2] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> import libxml2 > Traceback (most recent call last): > File "", line 1, in > File "/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2.py", line 1, in > import libxml2mod > ImportError: /nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2mod.so: undefined symbol: _PyVerify_fd I would try re-installing libxml2 *once* and then look for a package support list or email. -- Terry Jan Reedy From dieter at handshake.de Tue Feb 6 01:57:25 2018 From: dieter at handshake.de (dieter) Date: Tue, 06 Feb 2018 07:57:25 +0100 Subject: libxml2 installation/binding issue with Python 3.6.4 References: <68B69E4C07B04A488BA8528DF70CA165844150F7@fmsmsx116.amr.corp.intel.com> Message-ID: <87k1vqd2qy.fsf@handshake.de> "Priest, Matt" writes: > ... > I've successfully (?) installed Python 3.6.4 and libxml2, with the ultimate goal of installing GTK+ 3.22.0. You might also try "lxml" - which is an alternative for "libxml2". > However, I'm running into this error: > ... > import libxml2mod > ImportError: /nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2mod.so: undefined symbol: _PyVerify_fd This indicates a version mismatch between the C part of "libxml2" and something in your system - maybe the "libxml2" C library but more likely Python (given the name). Are you sure that "libxml2" is Python 3 ready? From Cecil at decebal.nl Tue Feb 6 03:42:47 2018 From: Cecil at decebal.nl (Cecil Westerhof) Date: Tue, 06 Feb 2018 09:42:47 +0100 Subject: From recovery.js to recoveryjsonlz4 References: <87372fpnz9.fsf@munus.decebal.nl> Message-ID: <87shaeo6ew.fsf@munus.decebal.nl> breamoreboy at gmail.com writes: > On Monday, February 5, 2018 at 1:28:16 PM UTC, Cecil Westerhof wrote: >> I have a script to get the number of windows and tabs that firefox >> uses. It always used a file recovery.js, but it changed to >> recovery.jsonlz4. >> >> Looking at the extension I would think it is an lz4 compressed file. >> But when I use: >> import lz4 >> >> I see that it is deprecated. How should I work with this file? >> > > I can't see anything here > http://python-lz4.readthedocs.io/en/stable/lz4.html to indicate that the > package is deprecated, although the lz4version function has been since > version 0.14. Further the latest 0.21.6 was released just yesterday, > 04/02/2018, so I'm guessing that you've just misread something. Well, when using 'import lz4' and giving tab in ipython, you get: deprecated library_version_number library_version_string lz4version VERSION version Looking at: http://python-lz4.readthedocs.io/en/stable/lz4.frame.html I tried to replace: loads(open(recover, 'rb').read().decode('utf-8')) With: import lz4.frame . . . lz4.frame.open(recover, 'rb').read().decode('utf-8') But this gives: -------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) in () ----> 1 dummy = lz4.frame.open(recover, 'rb').read().decode('utf-8') /usr/local/lib/python3.5/dist-packages/lz4/frame/__init__.py in read(self, size) 507 """ 508 self._check_can_read() --> 509 return self._buffer.read(size) 510 511 def read1(self, size=-1): /usr/lib/python3.5/_compression.py in read(self, size) 101 else: 102 rawblock = b"" --> 103 data = self._decompressor.decompress(rawblock, size) 104 if data: 105 break /usr/local/lib/python3.5/dist-packages/lz4/frame/__init__.py in decompress(self, data, max_length) 295 data, 296 max_length=max_length, --> 297 return_bytearray=self._return_bytearray, 298 ) 299 RuntimeError: LZ4F_decompress failed with code: ERROR_frameType_unknown What works is: dummy = lz4.frame.open(recover, 'rb') And this gives a: But when I then do: input = dummy.read() I get (as above): --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) in () ----> 1 input = dummy.read() /usr/local/lib/python3.5/dist-packages/lz4/frame/__init__.py in read(self, size) 507 """ 508 self._check_can_read() --> 509 return self._buffer.read(size) 510 511 def read1(self, size=-1): /usr/lib/python3.5/_compression.py in read(self, size) 101 else: 102 rawblock = b"" --> 103 data = self._decompressor.decompress(rawblock, size) 104 if data: 105 break /usr/local/lib/python3.5/dist-packages/lz4/frame/__init__.py in decompress(self, data, max_length) 295 data, 296 max_length=max_length, --> 297 return_bytearray=self._return_bytearray, 298 ) 299 RuntimeError: LZ4F_decompress failed with code: ERROR_frameType_unknown So how do I circumvent this error to read the file? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From alister.ware at ntlworld.com Tue Feb 6 05:23:26 2018 From: alister.ware at ntlworld.com (alister) Date: Tue, 06 Feb 2018 10:23:26 GMT Subject: "None" and "pass" References: <87vafbw74v.fsf@universite-de-strasbourg.fr.invalid> <85shafglpg.fsf@benfinney.id.au> Message-ID: On Tue, 06 Feb 2018 08:55:35 +1100, Chris Angelico wrote: > On Tue, Feb 6, 2018 at 8:39 AM, Ben Finney > wrote: >> Chris Angelico writes: >> >>> As one special case, I would accept this sort of code: >>> >>> def f(): >>> ... >>> >>> (three dots representing the special value Ellipsis) >>> >>> It's a great short-hand for "stub". >> >> I would not accept that. >> >> An even better way to write a stub function is to write a docstring: >> >> def frobnicate(): >> """ Frobnicate the spangule. """ >> >> A docstring, like any bare expression, is also a valid statement. >> Writing a docstring can be done immediately, because if you're writing >> a stub function you at least know the external interface of that >> function. >> >> > This is true, but I'd rather have something _under_ the docstring if > possible, and "..." works well for that. A docstring with nothing > underneath doesn't look like a stub - it looks like a failed edit or > something. Having a placeholder shows that it's intentional. > > ChrisA indeed and pass was implemented for precisely this usage why even think about possible alternatives -- Use an accordion. Go to jail. -- KFOG, San Francisco From rhodri at kynesim.co.uk Tue Feb 6 06:25:26 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 6 Feb 2018 11:25:26 +0000 Subject: "None" and "pass" In-Reply-To: References: <87vafbw74v.fsf@universite-de-strasbourg.fr.invalid> <85shafglpg.fsf@benfinney.id.au> Message-ID: On 06/02/18 10:23, alister via Python-list wrote: > On Tue, 06 Feb 2018 08:55:35 +1100, Chris Angelico wrote: > >> On Tue, Feb 6, 2018 at 8:39 AM, Ben Finney >> wrote: >>> Chris Angelico writes: >>> >>>> As one special case, I would accept this sort of code: >>>> >>>> def f(): >>>> ... >>>> >>>> (three dots representing the special value Ellipsis) >>>> >>>> It's a great short-hand for "stub". >>> >>> I would not accept that. >>> >>> An even better way to write a stub function is to write a docstring: >>> >>> def frobnicate(): >>> """ Frobnicate the spangule. """ >>> >>> A docstring, like any bare expression, is also a valid statement. >>> Writing a docstring can be done immediately, because if you're writing >>> a stub function you at least know the external interface of that >>> function. >>> >>> >> This is true, but I'd rather have something _under_ the docstring if >> possible, and "..." works well for that. A docstring with nothing >> underneath doesn't look like a stub - it looks like a failed edit or >> something. Having a placeholder shows that it's intentional. >> >> ChrisA > > indeed and pass was implemented for precisely this usage > why even think about possible alternatives None shall pass. (Seriously. I'm disappointed in all of you :-) -- Rhodri James *-* Kynesim Ltd From matt.priest at intel.com Tue Feb 6 10:34:28 2018 From: matt.priest at intel.com (Priest, Matt) Date: Tue, 6 Feb 2018 15:34:28 +0000 Subject: libxml2 installation/binding issue with Python 3.6.4 In-Reply-To: <87k1vqd2qy.fsf@handshake.de> References: <68B69E4C07B04A488BA8528DF70CA165844150F7@fmsmsx116.amr.corp.intel.com> <87k1vqd2qy.fsf@handshake.de> Message-ID: <68B69E4C07B04A488BA8528DF70CA165844152D0@fmsmsx116.amr.corp.intel.com> Dieter, I'm sure it's the Python version, I was hoping the "latest" set of the dependencies would get me there. I'll explore stepping that version back and let you know... Thank you, Matt -----Original Message----- From: Python-list [mailto:python-list-bounces+matt.priest=intel.com at python.org] On Behalf Of dieter Sent: Tuesday, February 6, 2018 12:57 AM To: python-list at python.org Subject: Re: libxml2 installation/binding issue with Python 3.6.4 "Priest, Matt" writes: > ... > I've successfully (?) installed Python 3.6.4 and libxml2, with the ultimate goal of installing GTK+ 3.22.0. You might also try "lxml" - which is an alternative for "libxml2". > However, I'm running into this error: > ... > import libxml2mod > ImportError: > /nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/ > python3.6/site-packages/libxml2mod.so: undefined symbol: _PyVerify_fd This indicates a version mismatch between the C part of "libxml2" and something in your system - maybe the "libxml2" C library but more likely Python (given the name). Are you sure that "libxml2" is Python 3 ready? -- https://mail.python.org/mailman/listinfo/python-list From drsalists at gmail.com Tue Feb 6 15:04:23 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 6 Feb 2018 12:04:23 -0800 Subject: Fwd: [Python-Dev] libxml2 installation/binding issue In-Reply-To: <68B69E4C07B04A488BA8528DF70CA165844150BC@fmsmsx116.amr.corp.intel.com> References: <68B69E4C07B04A488BA8528DF70CA165844150BC@fmsmsx116.amr.corp.intel.com> Message-ID: Perhaps look over (or use) http://stromberg.dnsalias.org/~strombrg/cpythons/ ? It defaults to building many versions of CPython, but can build just one if you prefer. It knows how to build GTK+ as well, for some newer versions of CPython. ---------- Forwarded message ---------- From: Priest, Matt Date: Mon, Feb 5, 2018 at 2:41 PM Subject: [Python-Dev] libxml2 installation/binding issue To: "python-dev at python.org" Hello, I am not sure if this is the correct place to post an issue/question like this, but here goes? I?ve successfully (?) installed Python 3.6.4 and libxml2, with the ultimate goal of installing GTK+ 3.22.0. However, I?m running into this error: python3 Python 3.6.4 (default, Feb 5 2018, 13:28:04) [GCC 4.7.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import libxml2 Traceback (most recent call last): File "", line 1, in File "/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2.py", line 1, in import libxml2mod ImportError: /nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2mod.so: undefined symbol: _PyVerify_fd Here are the details on the version, cflags, and ldflags. python3 --version ; Python 3.6.4 python3-config --cflags -I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/include/python3.6m -I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/include/python3.6m -Wno-unused-result -Wsign-compare -fPIC -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes python3-config ?ldflags; -L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/config-3.6m-x86_64-linux-gnu -L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib -lpython3.6m -lpthread -ldl -lutil -lrt -lm -Xlinker -export-dynamic Anyhelp or hint would be appreciated? Matt _______________________________________________ Python-Dev mailing list Python-Dev at python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/drsalists%40gmail.com From roel at roelschroeven.net Tue Feb 6 15:27:01 2018 From: roel at roelschroeven.net (Roel Schroeven) Date: Tue, 06 Feb 2018 21:27:01 +0100 Subject: Does anyone know ni? Message-ID: I'm having a look at py-iso8211 from https://sourceforge.net/projects/py-iso8211/ to see if I can get it to work without too much work. It uses a module 'ni', for example in __init__.py: """ ... """ # Make the above more easily available - if you do: # # import ni # ni.ni() # (currently needed (in Python 1.3) to start "ni" up) # import iso8211 # print iso8211.Intro # # then you should get the sensible result... Intro = __doc__ Is it something that was needed in very old Python versions to get access to the docstring? -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From ian.g.kelly at gmail.com Tue Feb 6 15:43:24 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 6 Feb 2018 13:43:24 -0700 Subject: Does anyone know ni? In-Reply-To: References: Message-ID: It was used for package support and is no longer needed from Python 1.5. http://legacy.python.org/doc/essays/packages.html On Tue, Feb 6, 2018 at 1:27 PM, Roel Schroeven wrote: > I'm having a look at py-iso8211 from > https://sourceforge.net/projects/py-iso8211/ to see if I can get it to work > without too much work. It uses a module 'ni', for example in __init__.py: > > > """ > ... > """ > > # Make the above more easily available - if you do: > # > # import ni > # ni.ni() # (currently needed (in Python 1.3) to start "ni" up) > # import iso8211 > # print iso8211.Intro > # > # then you should get the sensible result... > > Intro = __doc__ > > > Is it something that was needed in very old Python versions to get access to > the docstring? > > -- > The saddest aspect of life right now is that science gathers knowledge > faster than society gathers wisdom. > -- Isaac Asimov > > Roel Schroeven > > -- > https://mail.python.org/mailman/listinfo/python-list From roel at roelschroeven.net Tue Feb 6 15:52:53 2018 From: roel at roelschroeven.net (Roel Schroeven) Date: Tue, 06 Feb 2018 21:52:53 +0100 Subject: Does anyone know ni? In-Reply-To: References: Message-ID: Ian Kelly schreef op 6/02/2018 21:43: > It was used for package support and is no longer needed from Python > 1.5. http://legacy.python.org/doc/essays/packages.html Thanks! -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From israel at ravnalaska.net Tue Feb 6 16:12:26 2018 From: israel at ravnalaska.net (Israel Brewster) Date: Tue, 6 Feb 2018 12:12:26 -0900 Subject: Packaging uwsgi flask app for non-programmers? Message-ID: <605B7863-F542-4D54-A45A-1AD5284807F7@ravnalaska.net> I have been working on writing an Alexa skill which, as part of it, requires a local web server on the end users machine - the Alexa skill sends commands to this server, which runs them on the local machine. I wrote this local server in Flask, and run it using uwsgi, using a command like: "uwsgi serverconfig.ini". The problem is that in order for this to work, the end user must: 1) Install python 3.6 (or thereabouts) 2) Install a number of python modules, and 3) run a command line (from the appropriate directory) Not terribly difficult, but when I think of my target audience (Alexa users), I could easily see even these steps being "too complicated". I was looking at pyinstaller to create a simple double-click application, but it appears that pyinstaller needs a python script as the "base" for the application, whereas my "base" is uwsgi. Also, I do need to leave a config file accessible for the end user to be able to edit. Is there a way to use pyinstaller in this scenario, or perhaps some other option that might work better to package things up? ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From roel at roelschroeven.net Tue Feb 6 16:44:48 2018 From: roel at roelschroeven.net (Roel Schroeven) Date: Tue, 06 Feb 2018 22:44:48 +0100 Subject: How to work on a package Message-ID: I'm fairly comfortable writing Python code, but I only have experience writing scripts with perhaps a few supporting modules. Now I want to start writing a package, and I'm feeling a bit helpless: I'm not sure how to organize my work. In my way of thinking, I would have a working tree for the package (or even more than one), and also an installed version (once version 0.1 or so is ready). For example, let's say I'm working on luaparser (https://github.com/boolangery/py-lua-parser). There are tests in directory luaparser/tests, and I want to execute those tests. So I execute, for example: $ python3 -m unittest test_ast.py But that doesn't work: E ====================================================================== ERROR: test_ast (unittest.loader._FailedTest) ---------------------------------------------------------------------- ImportError: Failed to import test module: test_ast Traceback (most recent call last): File "/usr/lib/python3.5/unittest/loader.py", line 153, in loadTestsFromName module = __import__(module_name) File "/home/roel/temp/py-lua-parser/luaparser/tests/test_ast.py", line 1, in from luaparser.utils import tests ImportError: No module named 'luaparser' The test tries to import the global luaparser package (beause they use absoluate imports, which is the recommended way if I understand correctly), but that's not what I want: I want to test the local version in my working tree. I guess I could use something like $ PYTHONPATH=../.. python3 -m unittest test_ast.py but it feels like there should be a better way than manually specifying the module search path. How do people usually do this? Is there maybe a guide that explains the practical side of writing packages? -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From sjsumitj at gmail.com Tue Feb 6 22:42:37 2018 From: sjsumitj at gmail.com (Sum J) Date: Tue, 6 Feb 2018 19:42:37 -0800 (PST) Subject: How to set/update value in a xml file using requests in python Message-ID: My xml file is located in local network : http://192.168.43.109/DevMgmt/NetAppsDyn.xml Below is a part content of above xml I want to update : off I want to set value for 'ResourceUI' and 'Port' field in above xml. I have used below code : data = { 'ResourceURI':'web-proxy.xxx.yy.com', 'Port':8080 } URL = 'http://192.168.75.165/DevMgmt/NetAppsDyn.xml' # content type head = {'Content-type': 'text/xml'} # sending get request gr= requests.get(url=URL) print gr # sending put request r = requests.put(url=URL, data=data,headers=head) print r.status_code # extracting response text output_xml = r.text print("The op xml is:%s" % output_xml) Issue : The fields are not getting updated in xml using put request. I am able to see the response for get (request) , but for put request it is throwing errror code : 301 , resource has been moved permanently. From dieter at handshake.de Wed Feb 7 02:21:29 2018 From: dieter at handshake.de (dieter) Date: Wed, 07 Feb 2018 08:21:29 +0100 Subject: How to work on a package References: Message-ID: <878tc5mfie.fsf@handshake.de> Roel Schroeven writes: > I'm fairly comfortable writing Python code, but I only have experience > writing scripts with perhaps a few supporting modules. Now I want to > start writing a package, and I'm feeling a bit helpless: I'm not sure > how to organize my work. You may have a look at "https://packaging.python.org/" and "https://packaging.python.org/tutorials/distributing-packages/#requirements-for-packaging-and-distributing". > In my way of thinking, I would have a working tree for the package (or > even more than one), and also an installed version (once version 0.1 > or so is ready). > > For example, let's say I'm working on luaparser > (https://github.com/boolangery/py-lua-parser). There are tests in > directory luaparser/tests, and I want to execute those tests. So I > execute, for example: > > $ python3 -m unittest test_ast.py Likely, there are many ways to execute tests for your package. I am using "setuptools" for packaging (an extension of Python's standard "disutils"). Its "setup.py" supports the "test" command. This means, properly set up, I can run tests with "python setup.py test". From dieter at handshake.de Wed Feb 7 02:35:37 2018 From: dieter at handshake.de (dieter) Date: Wed, 07 Feb 2018 08:35:37 +0100 Subject: How to set/update value in a xml file using requests in python References: Message-ID: <874lmtmeuu.fsf@handshake.de> Sum J writes: > My xml file is located in local network : http://192.168.43.109/DevMgmt/NetAppsDyn.xml > > Below is a part content of above xml I want to update : > > > > > off > > > I want to set value for 'ResourceUI' and 'Port' field in above xml. > I have used below code : > > data = { > 'ResourceURI':'web-proxy.xxx.yy.com', > 'Port':8080 > } > > URL = 'http://192.168.75.165/DevMgmt/NetAppsDyn.xml' > > # content type > head = {'Content-type': 'text/xml'} > # sending get request > gr= requests.get(url=URL) > print gr > > # sending put request > r = requests.put(url=URL, data=data,headers=head) > print r.status_code > # extracting response text > output_xml = r.text > print("The op xml is:%s" % output_xml) > > Issue : The fields are not getting updated in xml using put request. I am able to see the response for get (request) , but for put request it is throwing errror code : 301 , resource has been moved permanently. This may not be a Python question. You are sending your data ("{'ResourceURI':'web-proxy.xxx.yy.com', 'Port':8080}) to the web service under "http://192.168.75.165/DevMgmt/NetAppsDyn.xml". Apparently, this web service does not interpret the data in the way you expect. Note, that you do not send an XML document to "NetAppsDyn.xml" (despite your `Content-type="text/xml"`). The typical (WevDAV) "put" would require that "data" contains a complete replacement for the object to be replaced, not just some modifications (of course, the object might have a specialized "put" which could support partial updates -- but apparently, in your current situation, this is either not the case or the "put" involved expects to get the replacement information in a different way). I would approach your task as follows: 1. Fetch the original XML 2. Use one of Python's XML libraries (I would use "lxml") to create the modified XML 3. "put" the modified XML to the server From tkadm30 at yandex.com Wed Feb 7 04:39:00 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 7 Feb 2018 04:39:00 -0500 Subject: Benchmarking Django on PyPy with unittest? Message-ID: Hi, is it possible to benchmark a django application? with unittest module in order to compare and measure the speed/latency of the django orm with sqlite3 against ZODB databases? i'm interested in comparing raw sqlite3 performance versus ZODB (schevo). i would like to make specific testsuite(s) for benchmarking django 1.11.7, django 2.0, pypy, etc. What do you think? Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From tkadm30 at yandex.com Wed Feb 7 05:26:12 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 7 Feb 2018 05:26:12 -0500 Subject: Benchmarking Django on PyPy with unittest? In-Reply-To: References: Message-ID: Also, i need to isolate and measure the speed of gevent loop engine (gevent.monkey), epoll, and python-specific asyncio coroutines. :-) Etienne Le 2018-02-07 ? 04:39, Etienne Robillard a ?crit?: > Hi, > > is it possible to benchmark a django application? with unittest module > in order to compare and measure the speed/latency of the django orm > with sqlite3 against ZODB databases? > i'm interested in comparing raw sqlite3 performance versus ZODB > (schevo). i would like to make specific testsuite(s) for benchmarking > django 1.11.7, django 2.0, pypy, etc. > > What do you think? > > Etienne > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From dhananjay.c.joshi at gmail.com Wed Feb 7 06:06:50 2018 From: dhananjay.c.joshi at gmail.com (Dhananjay) Date: Wed, 7 Feb 2018 19:06:50 +0800 Subject: Distance between point and a line passing through other two points Message-ID: Hello all, I have 3 points with coordinates (x0,y0,z0), (x1,y1,z1) and (x2,y2,z2). I also have a line joining points (x1,y1,z1) and (x2,y2,z2). For example, p0=[5.0, 5.0, 5.0] p1=[3.0, 3.0, 3.0] p2=[4.0, 4.0, 4.0] a = np.array(p0) b = np.array(p1) c = np.array(p2) I want to write a script that can calculate shortest distance d between point (x0,y0,z0) and the line((x1,y1,z1), (x2,y2,z2)). In other words, d = distance(a, line(b,c)) Since I have information of the coordinates of these points only, I am not sure how to put it into python script to get distance d. On searching Internet, some solutions are discussed for 2D coordinates (i.e. for (x0,y0), (x1,y1) and (x2,y2) ). However, I need solution for 3D coordinates. Any direction or suggestion would be great help. Thanking you in advance, -- Joshi From rosuav at gmail.com Wed Feb 7 06:39:03 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Feb 2018 22:39:03 +1100 Subject: Distance between point and a line passing through other two points In-Reply-To: References: Message-ID: On Wed, Feb 7, 2018 at 10:06 PM, Dhananjay wrote: > Hello all, > > I have 3 points with coordinates (x0,y0,z0), (x1,y1,z1) and (x2,y2,z2). > I also have a line joining points (x1,y1,z1) and (x2,y2,z2). > For example, > p0=[5.0, 5.0, 5.0] > p1=[3.0, 3.0, 3.0] > p2=[4.0, 4.0, 4.0] > > a = np.array(p0) > b = np.array(p1) > c = np.array(p2) > > I want to write a script that can calculate shortest distance d between > point (x0,y0,z0) and the line((x1,y1,z1), (x2,y2,z2)). > In other words, > d = distance(a, line(b,c)) > Since I have information of the coordinates of these points only, I am not > sure how to put it into python script to get distance d. > > On searching Internet, some solutions are discussed for 2D coordinates > (i.e. for (x0,y0), (x1,y1) and (x2,y2) ). > However, I need solution for 3D coordinates. > > Any direction or suggestion would be great help. > Thanking you in advance, With a line and a point not on that line, you can find exactly one plane that contains them all. (If the point is on the line, well, its distance is zero and nothing else matters. Your example points aren't particularly interesting, as the line through (3,3,3) and (4,4,4) also passes through (5,5,5).) So ultimately, this is solved the same way. Geometrically, you find the distance between a point and a line by dropping a perpendicular to that line and measuring that distance. Imagine drawing a triangle between the three points, dropping a perp, and measuring it off. Or type "distance point to line three dimensions" into your favourite web search engine and find a formula that way. Either way, once you have the mathematics settled, it should be fairly easy to translate that into Python. Enjoy! ChrisA From antoon.pardon at vub.be Wed Feb 7 06:49:28 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Wed, 7 Feb 2018 12:49:28 +0100 Subject: Distance between point and a line passing through other two points In-Reply-To: References: Message-ID: On 07-02-18 12:06, Dhananjay wrote: > Hello all, > > ... > I want to write a script that can calculate shortest distance d between > point (x0,y0,z0) and the line((x1,y1,z1), (x2,y2,z2)). > In other words, > d = distance(a, line(b,c)) > Since I have information of the coordinates of these points only, I am not > sure how to put it into python script to get distance d. > > On searching Internet, some solutions are discussed for 2D coordinates > (i.e. for (x0,y0), (x1,y1) and (x2,y2) ). > However, I need solution for 3D coordinates. Look at the vector solutions: https://en.wikipedia.org/wiki/Distance_from_a_point_to_a_line#Vector_formulation From Richard at Damon-Family.org Wed Feb 7 07:25:54 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Wed, 7 Feb 2018 07:25:54 -0500 Subject: Distance between point and a line passing through other two points In-Reply-To: References: Message-ID: On 2/7/18 6:06 AM, Dhananjay wrote: > Hello all, > > I have 3 points with coordinates (x0,y0,z0), (x1,y1,z1) and (x2,y2,z2). > I also have a line joining points (x1,y1,z1) and (x2,y2,z2). > For example, > p0=[5.0, 5.0, 5.0] > p1=[3.0, 3.0, 3.0] > p2=[4.0, 4.0, 4.0] > > a = np.array(p0) > b = np.array(p1) > c = np.array(p2) > > I want to write a script that can calculate shortest distance d between > point (x0,y0,z0) and the line((x1,y1,z1), (x2,y2,z2)). > In other words, > d = distance(a, line(b,c)) > Since I have information of the coordinates of these points only, I am not > sure how to put it into python script to get distance d. > > On searching Internet, some solutions are discussed for 2D coordinates > (i.e. for (x0,y0), (x1,y1) and (x2,y2) ). > However, I need solution for 3D coordinates. > > Any direction or suggestion would be great help. > Thanking you in advance, > > -- Joshi This sounds a lot more like a math problem then a python problem (unless python has a geometry package to use for this). The 3D solution is very close to the 2D version, if you draw the angle p0 -> p1 -> p2 and take the cross product of the vectors p1-p0 and p2-p1 the result will be a vector perpendicular to plane defined by p0, p1, p2 with a magnitude of |p1-p0| * |p2-p1| * sin(angle between line (p1,p0) and (p2,p1)) The distance you want is |p1-p0| * sin(angle between line (p1,p0) and (p2,p1)) With a bit of thought you should be able to get the answer. -- Richard Damon From israel at ravnalaska.net Wed Feb 7 11:58:46 2018 From: israel at ravnalaska.net (Israel Brewster) Date: Wed, 7 Feb 2018 07:58:46 -0900 Subject: Packaging uwsgi flask app for non-programmers? In-Reply-To: <543l7dp8bcvsnhem9ferlk70sm4masm6lf@4ax.com> References: <605B7863-F542-4D54-A45A-1AD5284807F7@ravnalaska.net> <543l7dp8bcvsnhem9ferlk70sm4masm6lf@4ax.com> Message-ID: > On Feb 6, 2018, at 8:24 PM, Dennis Lee Bieber wrote: > > On Tue, 6 Feb 2018 12:12:26 -0900, Israel Brewster > declaimed the following: > >> I have been working on writing an Alexa skill which, as part of it, requires a local web server on the end users machine - the Alexa skill sends commands to this server, which runs them on the local machine. I wrote this local server in Flask, and run it using uwsgi, using a command like: "uwsgi serverconfig.ini". >> > > > >> Not terribly difficult, but when I think of my target audience (Alexa users), I could easily see even these steps being "too complicated". I was looking at pyinstaller to create a simple double-click application, but it appears that pyinstaller needs a python script as the "base" for the application, whereas my "base" is uwsgi. Also, I do need to leave a config file accessible for the end user to be able to edit. Is there a way to use pyinstaller in this scenario, or perhaps some other option that might work better to package things up? > > Not mentioned is getting your end-user to possibly have to open up > fire-wall rules to allow INBOUND connections (even if, somehow, limited to > LAN -- don't want to leave a WAN port open). Not mentioned because it's not needed - I establish a ngrok tunnel to provide external https access to the local server. I just include the ngrok binary with my package, and run it using subprocess.Popen. Since it doesn't even require you to have an account to use it, that bypasses the need to set up port-forwards and firewall rules quite nicely. Also solves the problem of dynamic IP's without having to burden the end user with dyndns or the like - I just "register" the URL you get when connecting. Admittedly though, that was a large concern of mine until I was pointed to ngrok as a solution. Ideally, I'd just access across the local network, Alexa device to local machine, but that's not an option - at least, not yet. > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > -- > https://mail.python.org/mailman/listinfo/python-list From israel at ravnalaska.net Wed Feb 7 12:08:44 2018 From: israel at ravnalaska.net (Israel Brewster) Date: Wed, 7 Feb 2018 08:08:44 -0900 Subject: Packaging uwsgi flask app for non-programmers? In-Reply-To: <605B7863-F542-4D54-A45A-1AD5284807F7@ravnalaska.net> References: <605B7863-F542-4D54-A45A-1AD5284807F7@ravnalaska.net> Message-ID: On Feb 6, 2018, at 12:12 PM, Israel Brewster wrote: > > I have been working on writing an Alexa skill which, as part of it, requires a local web server on the end users machine - the Alexa skill sends commands to this server, which runs them on the local machine. I wrote this local server in Flask, and run it using uwsgi, using a command like: "uwsgi serverconfig.ini". > > The problem is that in order for this to work, the end user must: > > 1) Install python 3.6 (or thereabouts) > 2) Install a number of python modules, and > 3) run a command line (from the appropriate directory) > > Not terribly difficult, but when I think of my target audience (Alexa users), I could easily see even these steps being "too complicated". I was looking at pyinstaller to create a simple double-click application, but it appears that pyinstaller needs a python script as the "base" for the application, whereas my "base" is uwsgi. Also, I do need to leave a config file accessible for the end user to be able to edit. Is there a way to use pyinstaller in this scenario, or perhaps some other option that might work better to package things up? A related question, should a way to create a full package not be available, would be Is there a way to do a "local" (as in, in the same directory) install of Python3.6, and to do it in such a way as I could script it from the shell (or python, whatever)? The idea would then be to basically set up a fully self-contained virtualenv on the users machine, such that they just have to run a "setup.sh" script or the like. BTW, this would be on a Mac - my local skill server works using AppleScript, so it's not actually portable to other OS's :-P > > ----------------------------------------------- > Israel Brewster > Systems Analyst II > Ravn Alaska > 5245 Airport Industrial Rd > Fairbanks, AK 99709 > (907) 450-7293 > ----------------------------------------------- > > > > > -- > https://mail.python.org/mailman/listinfo/python-list From pengyu.ut at gmail.com Wed Feb 7 12:15:21 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Wed, 7 Feb 2018 11:15:21 -0600 Subject: Where is _sre.SRE_Match? Message-ID: Hi, I see _sre.SRE_Match is returned by re.match. But I don't find where it is defined. Does anybody know how to get its help page within python command line? Thanks. >>> import re >>> m = re.match('a', 'abc') >>> print type(m) >>> _sre.SRE_Match Traceback (most recent call last): File "", line 1, in NameError: name '_sre' is not defined -- Regards, Peng From rosuav at gmail.com Wed Feb 7 12:20:18 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 8 Feb 2018 04:20:18 +1100 Subject: Where is _sre.SRE_Match? In-Reply-To: References: Message-ID: On Thu, Feb 8, 2018 at 4:15 AM, Peng Yu wrote: > Hi, > > I see _sre.SRE_Match is returned by re.match. But I don't find where > it is defined. Does anybody know how to get its help page within > python command line? Thanks. > >>>> import re >>>> m = re.match('a', 'abc') >>>> print type(m) > >>>> _sre.SRE_Match > Traceback (most recent call last): > File "", line 1, in > NameError: name '_sre' is not defined > You can "import _sre" if you want access to that module, but the leading underscore is a strong indication that this isn't something you should be looking at - it's an implementation detail. Python 3 makes this type public: >>> import re >>> re.match('a', 'abc') >>> re.Match So if you truly need to examine this type, I suggest switching to a newer Python. ChrisA From breamoreboy at gmail.com Wed Feb 7 12:57:38 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 7 Feb 2018 09:57:38 -0800 (PST) Subject: Where is _sre.SRE_Match? In-Reply-To: References: Message-ID: On Wednesday, February 7, 2018 at 5:20:42 PM UTC, Chris Angelico wrote: > On Thu, Feb 8, 2018 at 4:15 AM, Peng Yu wrote: > > Hi, > > > > I see _sre.SRE_Match is returned by re.match. But I don't find where > > it is defined. Does anybody know how to get its help page within > > python command line? Thanks. > > > >>>> import re > >>>> m = re.match('a', 'abc') > >>>> print type(m) > > > >>>> _sre.SRE_Match > > Traceback (most recent call last): > > File "", line 1, in > > NameError: name '_sre' is not defined > > > > You can "import _sre" if you want access to that module, but the > leading underscore is a strong indication that this isn't something > you should be looking at - it's an implementation detail. > > Python 3 makes this type public: > > >>> import re > >>> re.match('a', 'abc') > > >>> re.Match > > > So if you truly need to examine this type, I suggest switching to a > newer Python. > > ChrisA I'm confused:- Python 3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import re >>> re.match('a', 'abc') <_sre.SRE_Match object; span=(0, 1), match='a'> >>> Perhaps not, https://bugs.python.org/issue30397 so 3.7 only. -- Kindestregards. Mark Lawrence. From rosuav at gmail.com Wed Feb 7 13:15:59 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 8 Feb 2018 05:15:59 +1100 Subject: Where is _sre.SRE_Match? In-Reply-To: References: Message-ID: On Thu, Feb 8, 2018 at 4:57 AM, wrote: > On Wednesday, February 7, 2018 at 5:20:42 PM UTC, Chris Angelico wrote: >> On Thu, Feb 8, 2018 at 4:15 AM, Peng Yu wrote: >> > Hi, >> > >> > I see _sre.SRE_Match is returned by re.match. But I don't find where >> > it is defined. Does anybody know how to get its help page within >> > python command line? Thanks. >> > >> >>>> import re >> >>>> m = re.match('a', 'abc') >> >>>> print type(m) >> > >> >>>> _sre.SRE_Match >> > Traceback (most recent call last): >> > File "", line 1, in >> > NameError: name '_sre' is not defined >> > >> >> You can "import _sre" if you want access to that module, but the >> leading underscore is a strong indication that this isn't something >> you should be looking at - it's an implementation detail. >> >> Python 3 makes this type public: >> >> >>> import re >> >>> re.match('a', 'abc') >> >> >>> re.Match >> >> >> So if you truly need to examine this type, I suggest switching to a >> newer Python. >> >> ChrisA > > I'm confused:- > > Python 3.6.3 (default, Oct 3 2017, 21:45:48) > [GCC 7.2.0] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> import re >>>> re.match('a', 'abc') > <_sre.SRE_Match object; span=(0, 1), match='a'> >>>> > > Perhaps not, https://bugs.python.org/issue30397 so 3.7 only. > Hmm, sorry for the confusion. I thought it landed earlier. (My default 'python3' is 3.7 so I didn't notice on testing.) Well, as you can see from the tracker issue, the classes were deliberately exposed in the way that I described. So... it's not just a matter of getting Python 3, but of getting *the latest* Python, in which what you request is easier. ChrisA From roel at roelschroeven.net Wed Feb 7 15:34:08 2018 From: roel at roelschroeven.net (Roel Schroeven) Date: Wed, 07 Feb 2018 21:34:08 +0100 Subject: How to work on a package In-Reply-To: <878tc5mfie.fsf@handshake.de> References: <878tc5mfie.fsf@handshake.de> Message-ID: dieter schreef op 7/02/2018 8:21: > Likely, there are many ways to execute tests for your package. > I am using "setuptools" for packaging (an extension > of Python's standard "disutils"). Its "setup.py" supports the "test" > command. This means, properly set up, I can run tests > with "python setup.py test". That can solve the testing issue, I guess, even though it feels weird to me that the most straightforward way doesn't work. But testing is not the only issue. Often I'd like to start the Python interpreter to load one of the modules in the package to try some things out, or write a little script to do the same. These things are very natural to me when writing Python code, so it seems very strange to me that there's no easy way when working on a packages. Don't other people do that? -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From rgaddi at highlandtechnology.invalid Wed Feb 7 16:24:27 2018 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 7 Feb 2018 13:24:27 -0800 Subject: How to work on a package In-Reply-To: References: <878tc5mfie.fsf@handshake.de> Message-ID: On 02/07/2018 12:34 PM, Roel Schroeven wrote: > dieter schreef op 7/02/2018 8:21: >> Likely, there are many ways to execute tests for your package. > >> I am using "setuptools" for packaging (an extension >> of Python's standard "disutils"). Its "setup.py" supports the "test" >> command. This means, properly set up, I can run tests >> with "python setup.py test". > > That can solve the testing issue, I guess, even though it feels weird to > me that the most straightforward way doesn't work. > > But testing is not the only issue. Often I'd like to start the Python > interpreter to load one of the modules in the package to try some things > out, or write a little script to do the same. These things are very > natural to me when writing Python code, so it seems very strange to me > that there's no easy way when working on a packages. > > Don't other people do that? > The state of Python packaging is... unfortunate. Improving, due to huge amounts of work by some very dedicated people, but still unfortunate. When I'm working on a module, the trick is to write a setup.py (using setuptools) from the very get-go. Before I write a single line of code, I've got a setup.py and the directory framework. Then you install the package using pip -e (or in practice --user -e). That's the missing piece. That way you can import your module from the interpreter, because it's now on the path, but its physical location is right there where you left it, complete with your VCS metadata and etc. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From roel at roelschroeven.net Wed Feb 7 18:11:40 2018 From: roel at roelschroeven.net (Roel Schroeven) Date: Thu, 08 Feb 2018 00:11:40 +0100 Subject: How to work on a package In-Reply-To: References: <878tc5mfie.fsf@handshake.de> Message-ID: Rob Gaddi schreef op 7/02/2018 22:24: > On 02/07/2018 12:34 PM, Roel Schroeven wrote: >> dieter schreef op 7/02/2018 8:21: >>> Likely, there are many ways to execute tests for your package. >>> I am using "setuptools" for packaging (an extension >>> of Python's standard "disutils"). Its "setup.py" supports the "test" >>> command. This means, properly set up, I can run tests >>> with "python setup.py test". >> That can solve the testing issue, I guess, even though it feels weird to >> me that the most straightforward way doesn't work. >> >> But testing is not the only issue. Often I'd like to start the Python >> interpreter to load one of the modules in the package to try some things >> out, or write a little script to do the same. These things are very >> natural to me when writing Python code, so it seems very strange to me >> that there's no easy way when working on a packages. >> >> Don't other people do that? >> > > The state of Python packaging is... unfortunate. Improving, due to huge > amounts of work by some very dedicated people, but still unfortunate. > > When I'm working on a module, the trick is to write a setup.py (using > setuptools) from the very get-go. Before I write a single line of code, > I've got a setup.py and the directory framework. > > Then you install the package using pip -e (or in practice --user -e). > That's the missing piece. That way you can import your module from the > interpreter, because it's now on the path, but its physical location is > right there where you left it, complete with your VCS metadata and etc. That seems like a workable solution. I'll try that out. Thanks! Best regards, Roel -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From grant.b.edwards at gmail.com Wed Feb 7 18:17:10 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 7 Feb 2018 23:17:10 +0000 (UTC) Subject: How to work on a package References: <878tc5mfie.fsf@handshake.de> Message-ID: On 2018-02-07, Rob Gaddi wrote: > When I'm working on a module, the trick is to write a setup.py (using > setuptools) from the very get-go. Before I write a single line of code, > I've got a setup.py and the directory framework. > > Then you install the package using pip -e (or in practice --user -e). > That's the missing piece. That way you can import your module from the > interpreter, because it's now on the path, but its physical location is > right there where you left it, complete with your VCS metadata and etc. How do you work on a package that must remain installed and usable the whole time you're working on it? IOW, only specific test apps or apps run in a specific directory should get the "in-progress" foo module when they do an "import foo". -- Grant Edwards grant.b.edwards Yow! Am I in GRADUATE at SCHOOL yet? gmail.com From rgaddi at highlandtechnology.invalid Wed Feb 7 18:46:58 2018 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 7 Feb 2018 15:46:58 -0800 Subject: How to work on a package In-Reply-To: References: <878tc5mfie.fsf@handshake.de> Message-ID: On 02/07/2018 03:17 PM, Grant Edwards wrote: > On 2018-02-07, Rob Gaddi wrote: > >> When I'm working on a module, the trick is to write a setup.py (using >> setuptools) from the very get-go. Before I write a single line of code, >> I've got a setup.py and the directory framework. >> >> Then you install the package using pip -e (or in practice --user -e). >> That's the missing piece. That way you can import your module from the >> interpreter, because it's now on the path, but its physical location is >> right there where you left it, complete with your VCS metadata and etc. > > How do you work on a package that must remain installed and usable the > whole time you're working on it? > > IOW, only specific test apps or apps run in a specific directory > should get the "in-progress" foo module when they do an "import foo". > Achievable with a virtualenv, but now the process is even that much more complicated. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From drsalists at gmail.com Wed Feb 7 19:10:21 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Wed, 7 Feb 2018 16:10:21 -0800 Subject: Benchmarking Django on PyPy with unittest? In-Reply-To: References: Message-ID: You could probably use the "requests" module to time how long various operations take in your Django website. On Wed, Feb 7, 2018 at 2:26 AM, Etienne Robillard wrote: > Also, i need to isolate and measure the speed of gevent loop engine > (gevent.monkey), epoll, and python-specific asyncio coroutines. :-) > > Etienne > > > > Le 2018-02-07 ? 04:39, Etienne Robillard a ?crit : >> >> Hi, >> >> is it possible to benchmark a django application with unittest module in >> order to compare and measure the speed/latency of the django orm with >> sqlite3 against ZODB databases? >> i'm interested in comparing raw sqlite3 performance versus ZODB (schevo). >> i would like to make specific testsuite(s) for benchmarking django 1.11.7, >> django 2.0, pypy, etc. >> >> What do you think? >> >> Etienne >> > > -- > Etienne Robillard > tkadm30 at yandex.com > https://www.isotopesoftware.ca/ > > -- > https://mail.python.org/mailman/listinfo/python-list From dieter at handshake.de Thu Feb 8 02:45:35 2018 From: dieter at handshake.de (dieter) Date: Thu, 08 Feb 2018 08:45:35 +0100 Subject: How to work on a package References: <878tc5mfie.fsf@handshake.de> Message-ID: <87r2pvgc0w.fsf@handshake.de> Roel Schroeven writes: > dieter schreef op 7/02/2018 8:21: > ... >> Likely, there are many ways to execute tests for your package. > >> I am using "setuptools" for packaging (an extension >> of Python's standard "disutils"). Its "setup.py" supports the "test" >> command. This means, properly set up, I can run tests >> with "python setup.py test". > > That can solve the testing issue, I guess, even though it feels weird > to me that the most straightforward way doesn't work. > > But testing is not the only issue. Often I'd like to start the Python > interpreter to load one of the modules in the package to try some > things out, or write a little script to do the same. These things are > very natural to me when writing Python code, so it seems very strange > to me that there's no easy way when working on a packages. I use a "virtualenv" for this: it is like a private Python environment with its own set of packages. The "setup" from "setuptools" not only supports a "test" command but also a "develop" command. This "links" the package (source) into Python's (the one you used to run "setup.py"; i.e. usually the one of a "virtualenv") library and thereby makes it available for this Python. From tkadm30 at yandex.com Thu Feb 8 04:31:38 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Thu, 8 Feb 2018 04:31:38 -0500 Subject: Benchmarking Django on PyPy with unittest? In-Reply-To: References: Message-ID: Hi Dan, Thank you for your reply. I never used the requests module, but it looks like a very good pick to test the Django wsgi handler. :-) Currently my tests/benchmarks folder looks like this: tests/benchmarks/lib tests/benchmarks/lib/django_sqlite # Django 1.11 specific project directory tests/benchmarks/lib/django_sqlite/myapp # Test app for benchmarking templates, views, and core django api tests/benchmarks/lib/django_sqlite/polls # Tutorial app for benchmarking django orm/sqlite on pypy and cpython tests/benchmarks/uwsgi # Testsuite for uWSGI tests/benchmarks/django1 # Testsuite for Django 1.11 tests/benchmarks/django2 # Testsuite for Django 2 To run the benchmarks testsuite on pypy: $ cd tests $ source ./djangorc $ pypy ./run.py -C benchmarks/uwsgi # Run testsuite for the uwsgi handler $ pypy ./run.py -C benchmarks/django1 # Run testsuite for the django 1.11 api $ pypy ./run.py -C benchmarks/django2 # Run testsuite for the django 2.0 api Ideally, i would like to compile benchmarks data, but I have not yet understood how to do this. :-) Cheers, Etienne 2018-02-07 ? 19:10, Dan Stromberg a ?crit?: > You could probably use the "requests" module to time how long various > operations take in your Django website. > > On Wed, Feb 7, 2018 at 2:26 AM, Etienne Robillard wrote: >> Also, i need to isolate and measure the speed of gevent loop engine >> (gevent.monkey), epoll, and python-specific asyncio coroutines. :-) >> >> Etienne >> >> >> >> Le 2018-02-07 ? 04:39, Etienne Robillard a ?crit : >>> Hi, >>> >>> is it possible to benchmark a django application with unittest module in >>> order to compare and measure the speed/latency of the django orm with >>> sqlite3 against ZODB databases? >>> i'm interested in comparing raw sqlite3 performance versus ZODB (schevo). >>> i would like to make specific testsuite(s) for benchmarking django 1.11.7, >>> django 2.0, pypy, etc. >>> >>> What do you think? >>> >>> Etienne >>> >> -- >> Etienne Robillard >> tkadm30 at yandex.com >> https://www.isotopesoftware.ca/ >> >> -- >> https://mail.python.org/mailman/listinfo/python-list -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From grant.b.edwards at gmail.com Thu Feb 8 10:16:37 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 8 Feb 2018 15:16:37 +0000 (UTC) Subject: How to work on a package References: <878tc5mfie.fsf@handshake.de> Message-ID: On 2018-02-07, Rob Gaddi wrote: > On 02/07/2018 03:17 PM, Grant Edwards wrote: > >> How do you work on a package that must remain installed and usable the >> whole time you're working on it? >> >> IOW, only specific test apps or apps run in a specific directory >> should get the "in-progress" foo module when they do an "import foo". > > Achievable with a virtualenv, but now the process is even that much more > complicated. Ouch. I was afraid that was the answer. There is nothing that corresponds to LD_LIBRARY_PATH? -- Grant Edwards grant.b.edwards Yow! Now, let's SEND OUT at for QUICHE!! gmail.com From mal at europython.eu Thu Feb 8 10:31:18 2018 From: mal at europython.eu (M.-A. Lemburg) Date: Thu, 8 Feb 2018 16:31:18 +0100 Subject: EuroPython 2018: Getting ready Message-ID: <3880c208-a5cb-98d5-6f12-9e9f04151cbb@europython.eu> We are excited to announce the launch of the EuroPython 2018 website: * https://ep2018.europython.eu/ * The EuroPython conference will take place in sunny Edinburgh, Scotland, this year, from July 23-29 2018. It?s a great time of year to visit Edinburgh with 16 hours of daylight, and the festival season in full flow, so come and join us. This is just one week before the famous Edinburgh Fringe Festival and the Turing Festival, so you can extend your stay a little longer in Edinburgh, or head for the Highlands to enjoy the amazing mountains and lochs. EuroPython 2018 - The European Python Conference ------------------------------------------------ Here?s an overview of what you can expect in Edinburgh: - We will start with Workshops and Training Sessions on Monday and Tuesday. - The main 3 conference days follow, packed with keynotes, talks, exhibition, help desks, interactive sessions, panels and poster sessions. - The two weekend days after the conference, July 28 and 29, are reserved for sprints (hackathons). Overall, we will again have 7 days worth of great Python content, arranged in over 120 sessions, waiting for you. The venue is the Edinburgh International Conference Centre, in central Edinburgh, just on the edge of the historic Old Town. In short: - Monday, Tuesday, July 23-24: Workshops and Training - Wednesday - Friday, July 25-27: Conference talks, keynotes, training - Saturday, Sunday, July 28-29: Sprints Our Sponsors ------------ All this would not be possible without the generous help of our launch sponsors. If your company would be interested in sponsoring the 17th EuroPython please contact sponsoring at europython.eu. Sponsoring EuroPython guarantees you highly targeted visibility and the opportunity to present yourself and your company in a professional and innovative environment. You?ll have an unique opportunity to meet many Python-enthusiastic developers, users and professionals. As a sponsor of EuroPython 2018, you will directly help to promote the work of a great open-source community and help further its development. EuroPython 2018 is the 17th EuroPython conference. The conference tours throughout Europe. It so far has had stops in Belgium, Sweden, Lithuania, United Kingdom, Italy, Germany and Basque Country/Spain, growing from initially 240 attendees to well over 1200. In the coming days, we will announce the start of the Call for Proposals and Early Bird Ticket sales. Please watch our EuroPython blog for updates. https://ep2018.europython.eu/ Enjoy, -- EuroPython 2018 Team https://ep2018.europython.eu/ https://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/961616731482488833 Thanks. From drsalists at gmail.com Thu Feb 8 11:45:01 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 8 Feb 2018 08:45:01 -0800 Subject: [Python-Dev] How to set/update value in a xml file using requests in python In-Reply-To: References: Message-ID: This is more relevant to python-list than python-dev. I've added python-list to the To header. Gmail doesn't appear to allow setting a reply-to for a single message, so I've not set that; please, when replying, drop python-dev from the to: header. You'll likely want to set up some kind of REST endpoint to allow updating your xml file. Imagine if anyone could change anyone else's xml files - it'd be pretty chaotic. Perhaps flask would be appropriate? http://flask.pocoo.org/docs/0.12/quickstart/ You likely will want a PUT or a POST HTTP verb: https://stackoverflow.com/questions/107390/whats-the-difference-between-a-post-and-a-put-http-request HTH. On Thu, Feb 8, 2018 at 12:38 AM, Sum J wrote: > My xml file is located in local network: > > http://192.168.43.109/DevMgmt/NetAppsDyn.xml > > Below is a part content of above xml I want to update : > > > > > > > off > > > > > I want to set the value for 'ResourceUI' and 'Port' field in above xml. > > I have used below code : > > import requests > data = { > 'ResourceURI':'web-proxy.xxx.yy.com', > 'Port':8080 > } > > URL = 'http://192.168.75.165/DevMgmt/NetAppsDyn.xml' > > # content type > head = {'Content-type': 'text/xml'} > # sending get request > gr= requests.get(url=URL) > print gr > > # sending put request > r = requests.put(url=URL, data=data,headers=head) > print r.status_code > # extracting response text > output_xml = r.text > print("The op xml is:%s" % output_xml) > > Issue : The fields are not getting updated in xml using put request. I am > able to see the response for get (request) , but for put request it is > throwing errror code : 301 , resource has been moved permanently. > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/drsalists%40gmail.com > From none at invalid.com Thu Feb 8 12:46:22 2018 From: none at invalid.com (mm0fmf) Date: Thu, 08 Feb 2018 17:46:22 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <1871443595@f38.n261.z1.binkp.net> <4265150478@f38.n261.z1.binkp.net> Message-ID: On 08/02/2018 17:36, Gilmeh Serda wrote: > On Sat, 03 Feb 2018 04:33:36 +1200, breamoreboy wrote: > >>> When trying to access comp.lang.idl-pvwave, a message is now displayed, >> stating that the group owner needs to remove the spam, and can then >> apply to Google in order to have access reinstated. > > Just as I have always suspected: Google are run by idiots! > But very, very, very rich idiots. From larry.martell at gmail.com Thu Feb 8 12:47:48 2018 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 8 Feb 2018 12:47:48 -0500 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <1871443595@f38.n261.z1.binkp.net> <4265150478@f38.n261.z1.binkp.net> Message-ID: On Thu, Feb 8, 2018 at 12:36 PM, Gilmeh Serda wrote: > On Sat, 03 Feb 2018 04:33:36 +1200, breamoreboy wrote: > >>> When trying to access comp.lang.idl-pvwave, a message is now displayed, >> stating that the group owner needs to remove the spam, and can then >> apply to Google in order to have access reinstated. > > Just as I have always suspected: Google are run by idiots! Very, very rich idiots. From breamoreboy at gmail.com Thu Feb 8 17:35:08 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 8 Feb 2018 14:35:08 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: <2802702295@f38.n261.z1.binkp.net> References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> Message-ID: <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> On Sunday, February 4, 2018 at 12:15:16 AM UTC, pyotr filipivich wrote: > Those of us who do not use google-groups may not notice the loss > of the google groupies. > -- > pyotr filipivich > Next month's Panel: Graft - Boon or blessing? This topic has been hidden because you reported it for abuse. (+39 more) Re: This newsgroup (comp.lang.python) may soon be blocked by Google Gro (2) By pyotr filipivich 4 posts 14 views 5:48 PM How to set/update value in a xml file using requests in python (5) By Sum J 5 posts 21 views 4:45 PM EuroPython 2018: Getting ready (1) By M.-A. Lemburg 1 post 3 views 3:40 PM How to work on a package (11) By Roel Schroeven 11 posts 34 views 3:19 PM This topic has been hidden because you reported it for abuse. (+25 more) Benchmarking Django on PyPy with unittest? (1) By Etienne Robillard 6 posts 18 views 9:33 AM This topic has been hidden because you reported it for abuse. (+18 more) Does anyone know ni? (6) By Roel Schroeven 6 posts 17 views 12:15 AM This topic has been hidden because you reported it for abuse. +1 more The above is taken from my small laptop screen showing just how useless google groups is, so it might as well be blown away completely, or is it impossible for the PSF to take on google? The most that I've ever whacked in one hit was +55 more, all of it the Case Solutions/Test Bank crap. Just saying. -- Kindest regards. Mark Lawrence. From porton at narod.ru Thu Feb 8 17:41:34 2018 From: porton at narod.ru (Victor Porton) Date: Fri, 09 Feb 2018 00:41:34 +0200 Subject: $srcdir and $datadir Message-ID: In GNU software written in C $srcdir and $datadir are accessible to C code through generated config.h file. What is the right way to config directories for a Python program? -- Victor Porton - http://portonvictor.org From ben+python at benfinney.id.au Thu Feb 8 18:05:54 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 09 Feb 2018 10:05:54 +1100 Subject: $srcdir and $datadir References: Message-ID: <854lmrf5f1.fsf@benfinney.id.au> Victor Porton writes: > In GNU software written in C $srcdir and $datadir are accessible to C > code through generated config.h file. For what purpose? Given that the source may not be at that location after the program is compiled ? especially, after the program is moved to a different machine ? what meaning does ?$srcdir? have when the program is running? What ?data directory? is specified by ?$datadir?, and why is it assumed there is exactly one? > What is the right way to config directories for a Python program? We'll need to know what those concepts mean, to be able to discuss the equivalent (if any) in a Python environment. -- \ ?Shepherds ? look after their sheep so they can, first, fleece | `\ them and second, turn them into meat. That's much more like the | _o__) priesthood as I know it.? ?Christopher Hitchens, 2008-10-29 | Ben Finney From rosuav at gmail.com Thu Feb 8 18:33:03 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 9 Feb 2018 10:33:03 +1100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> Message-ID: On Fri, Feb 9, 2018 at 9:35 AM, wrote: > On Sunday, February 4, 2018 at 12:15:16 AM UTC, pyotr filipivich wrote: > >> Those of us who do not use google-groups may not notice the loss >> of the google groupies. >> -- >> pyotr filipivich >> Next month's Panel: Graft - Boon or blessing? > > This topic has been hidden because you reported it for abuse. (+39 more) > Re: This newsgroup (comp.lang.python) may soon be blocked by Google Gro (2) > By pyotr filipivich > 4 posts 14 views > 5:48 PM > How to set/update value in a xml file using requests in python (5) > By Sum J > 5 posts 21 views > 4:45 PM > EuroPython 2018: Getting ready (1) > By M.-A. Lemburg > 1 post 3 views > 3:40 PM > How to work on a package (11) > By Roel Schroeven > 11 posts 34 views > 3:19 PM > This topic has been hidden because you reported it for abuse. (+25 more) > Benchmarking Django on PyPy with unittest? (1) > By Etienne Robillard > 6 posts 18 views > 9:33 AM > This topic has been hidden because you reported it for abuse. (+18 more) > Does anyone know ni? (6) > By Roel Schroeven > 6 posts 17 views > 12:15 AM > This topic has been hidden because you reported it for abuse. +1 more > > The above is taken from my small laptop screen showing just how useless google groups is, so it might as well be blown away completely, or is it impossible for the PSF to take on google? > > The most that I've ever whacked in one hit was +55 more, all of it the Case Solutions/Test Bank crap. The PSF doesn't *need* to take on Google, because there are other newsgroup hosting sites that have less spam. Switch to one of those, and voila, less spam in your inbox. ChrisA From breamoreboy at gmail.com Thu Feb 8 18:40:57 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Thu, 8 Feb 2018 23:40:57 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> Message-ID: On 08/02/18 23:33, Chris Angelico wrote: > > The PSF doesn't *need* to take on Google, because there are other > newsgroup hosting sites that have less spam. Switch to one of those, > and voila, less spam in your inbox. > > ChrisA > I don't see any spam in my inbox as I read on gmane, I'm pointing out the disgraceful state of gg and why it should be dropped as it's less than useless. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From python.list at tim.thechases.com Thu Feb 8 20:56:40 2018 From: python.list at tim.thechases.com (Tim Chase) Date: Thu, 8 Feb 2018 19:56:40 -0600 Subject: virtualenvwrapper under OpenBSD's ksh & FreeBSD's /bin/sh In-Reply-To: <20180202131024.24378879@bigbox.christie.dr> References: <20180202131024.24378879@bigbox.christie.dr> Message-ID: <20180208195640.3080b455@bigbox.christie.dr> Giving a nudge here. I've tried the below process with both OpenBSD's stock ksh and FreeBSD's stock /bin/sh as my shell and both seem to have similar errors (the FreeBSD error is less precise about the line location or the actual error: /home/tim/.local/bin/virtualenvwrapper.sh: ${....}: Bad substitution but I suspect it's the same). Is there a way to get `pip install --user virtualenvwrapper` to pull in a version that supports either ksh or traditional /bin/sh instead of being yoked to bash/zsh? Based on that bitbucket.org/dhellmann link, it sounds like it should at least support ksh. Thanks, -tkc On 2018-02-02 13:10, Tim Chase wrote: > Under a new user account with ksh (the default) as the shell I > issued the following: > > $ pip3 install --user virtualenvwrapper > Successfully installed pbr-3.1.1 six-1.11.0 stevedore-1.28.0 > virtualenv-clone-0.2.6 virtualenvwrapper-4.8.2 $ export > WORKON_HOME=~/code/virtualenvs $ mkdir -p $WORKON_HOME > > Good so far. Based on > > https://bitbucket.org/dhellmann/virtualenvwrapper-hg > > it sounds like ksh should be supported. However when I try to > enable it, I get: > > $ . ~/.local/bin/virtualenvwrapper.sh > ksh: /home/tim/.local/bin/virtualenvwrapper.sh[97]: ${.sh.file}": > bad substitution > > The line in question reads > > virtualenvwrapper.sh: export > VIRTUALENVWRAPPER_SCRIPT="${.sh.file}" > > though it's not present in the latest tip version of the source. I > tried pulling in that one virtualenvwrapper.sh file from the tip to > see if that would remedy the issue but it complains > > $ . ~/tmp/virtualenvwrapper.sh > ksh: /home/tim/tmp/virtualenvwrapper.sh[247]: syntax error: `(' > unexpected > > on this line > > COMPREPLY=( $(compgen -W "`virtualenvwrapper_show_workon_options`" > -- ${cur}) ) > > Is there something I'm missing or need to do to get pip (pip3.6) to > pull in a working version of virtualenvwrapper for ksh? > > Thanks, > > -tkc > > > > -- > https://mail.python.org/mailman/listinfo/python-list From steve+comp.lang.python at pearwood.info Fri Feb 9 00:00:33 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 9 Feb 2018 05:00:33 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> Message-ID: On Thu, 08 Feb 2018 23:40:57 +0000, Mark Lawrence wrote: > I don't see any spam in my inbox as I read on gmane, I'm pointing out > the disgraceful state of gg and why it should be dropped as it's less > than useless. Who is it that you think should "drop" Google Groups? If you mean individual people, it is their choice to use GG or not. I understand why some people might choose to use GG in general: it gives a reasonably discoverable web interface to Usenet. Many people don't have access to Usenet, or even know it exists, so using GG is an obvious (if sub-par) alternative. (My own ISP appears to have dropped Usenet access, *yet again*. Each time they do so, there are enough complaints that they bring it back, but they simply don't stop trying.) If you mean that the PSF should drop GG, that's not the PSF's choice to make. They don't send data specifically to GG and they can't shut off the pipe except by shutting it down for *everyone*. Google simply collects the data and mirrors it. The only one who can stop that is Google themselves. -- Steve From dallasdisabilityattorney at gmail.com Fri Feb 9 00:04:37 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Thu, 8 Feb 2018 21:04:37 -0800 (PST) Subject: Extracting data from ython dictionary object Message-ID: <27be3bcf-a064-425f-9b11-edbead597db7@googlegroups.com> I am new to Python. I am trying to extract text from the bookmarks in a PDF file that would provide the data for a Word template merge. I have gotten down to a string of text pulled out of the list object that I got from using PyPDF2 module. I am stuck on now to get the data out of the string that I need. I am calling it a string, but Python is recognizing as a dictionary object. Here is the string: {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} What a want is the following to end up as fields on my Word template merge: MedSourceFirstName: "John" MedSourceLastName: "Milani" MedSourceLastTreatment: "05/28/2014" If I use keys() on the dictionary I get this: ['/Title', '/Page', '/Type']I was hoping "Src" and Tmt Dt." would be treated as keys. Seems like the key/value pair of a dictionary would translate nicely to fieldname and fielddata for a Word document merge. Here is my code so far. [python]import PyPDF2 pdfFileObj=open('x.pdf','rb') pdfReader=PyPDF2.PdfFileReader(pdfFileObj) MyList=pdfReader.getOutlines() MyDict=(MyList[-1][0]) print(isinstance(MyDict,dict)) print(MyDict) print(list(MyDict.keys()))[/python] I get this output in Sublime Text: True {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} ['/Title', '/Page', '/Type'] [Finished in 0.4s] Thank you in advance for any suggestions. From dieter at handshake.de Fri Feb 9 02:07:04 2018 From: dieter at handshake.de (dieter) Date: Fri, 09 Feb 2018 08:07:04 +0100 Subject: Extracting data from ython dictionary object References: <27be3bcf-a064-425f-9b11-edbead597db7@googlegroups.com> Message-ID: <8760761w13.fsf@handshake.de> Stanley Denman writes: > I am new to Python. I am trying to extract text from the bookmarks in a PDF file that would provide the data for a Word template merge. I have gotten down to a string of text pulled out of the list object that I got from using PyPDF2 module. I am stuck on now to get the data out of the string that I need. I am calling it a string, but Python is recognizing as a dictionary object. > > Here is the string: > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > What a want is the following to end up as fields on my Word template merge: > MedSourceFirstName: "John" > MedSourceLastName: "Milani" > MedSourceLastTreatment: "05/28/2014" > > If I use keys() on the dictionary I get this: > ['/Title', '/Page', '/Type']I was hoping "Src" and Tmt Dt." would be treated as keys. Seems like the key/value pair of a dictionary would translate nicely to fieldname and fielddata for a Word document merge. Here is my code so far. A Python "dict" is a mapping of keys to values. Its "keys" method gives you the keys (as you have used above). The subscription syntax ("[]"; e.g. "pdf_info['/Title']") allows you to access the value associated with "". In your case, relevant information is coded inside the values themselves. You will need to extract this information yourself. Python's "re" module might be of help (see the "library reference", for details). From mail at timgolden.me.uk Fri Feb 9 03:49:57 2018 From: mail at timgolden.me.uk (Tim Golden) Date: Fri, 9 Feb 2018 08:49:57 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> Message-ID: On 09/02/2018 05:00, Steven D'Aprano wrote: > On Thu, 08 Feb 2018 23:40:57 +0000, Mark Lawrence wrote: > >> I don't see any spam in my inbox as I read on gmane, I'm pointing out >> the disgraceful state of gg and why it should be dropped as it's less >> than useless. > > Who is it that you think should "drop" Google Groups? If Mark means: the mailing list gateway should automatically drop posts originating from GG, then this wouldn't really buy us anything we don't have. I think it's unlikely that anyone still following this thread isn't aware of the situation, but just in case: We have a newsgroup (comp.lang.python) and a mailing list (python-list) which are mirrored to each other by virtue of the Mailman gateway (ie the mailing list handles both sides of the mirror). Google Groups offers a mail & web interface to the newsgroup Gmane offers a newsgroup interface to the mailing list The newsgroup is entirely unmoderated and unfiltered The mailing list is moderated and filtered in different ways: * We block (hold or discard) mailing list posts by subscriber * We block inbound usenet posts by header (usually sender email but also, eg, ISP) * We have a Spambayes instance running which injects a spam score header and we block on that header * We have a few other more specific filters in place for various issues which have arisen over the years By and large, this combination of filters, coupled with the work of the moderation and postmaster team, is enough to keep the list "clean" of manifest spam and so on. Obviously, the first instance of spam might slip past all our filters, in which case we're on the back foot and have to react. On the whole, though, we catch most things. So dropping GG altogether would probably not add very much, as we're generally blocking undesirable posts from there and we'd rather not block genuine posts which happen to be made through the relative convenience of the GG interface. Can I call out for thanks here the people who have worked behind the scenes for quite a long time to keep this list running smoothly, including: Skip Montanaro who manages the Spambayes setup; Mark Sapiro, the Mailman 2 dev lead who looks after all the python.org Mailman2 instances; Ralf Hildebrandt who acts as Postmaster; of course Barry Warsaw who created Mailman (and this list, I believe) and still contributes when we need to discuss matters as list owners; of course Ethan Furman, my fellow list moderator; and doubtless others whom I've forgotten. The fact that the list continues to be readable and useful is down in large part to them. TJG From john_ladasky at sbcglobal.net Fri Feb 9 06:23:59 2018 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Fri, 9 Feb 2018 03:23:59 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> Message-ID: On Friday, February 9, 2018 at 12:50:16 AM UTC-8, Tim Golden wrote: [snip and rearrange] > So dropping GG altogether would probably not add very much, as we're > generally blocking undesirable posts from there and we'd rather not > block genuine posts which happen to be made through the relative > convenience of the GG interface. I thank you for this point of view. I am using Google Groups. I migrated here back in the days of Deja News. I don't like having mailing lists streaming into my inbox, having to hunt around to unsubscribe. I want a newsreader with integrated posting capability which is separate from my inbox. > Gmane offers a newsgroup interface to the mailing list I haven't visited GMane in a few years, but I found it difficult to navigate. In particular, I found searching to be cumbersome. Weren't the archives broken into 30-day blocks? I just tried GMane again two minutes ago. I can't confirm my recollections, but right now what I'm seeing is worse. If you follow this link right now... (http://gmane.org/find.php?list=comp.lang.python)... you get this: "Not all of Gmane is back yet - We're working hard to restore everything" And if you follow the link labeled "The latest news is at Gmane News" at the bottom of that page, it takes you here... (http://home.gmane.org/)... and the top blog post discussing site repairs is dated September 2016! I'm not too excited about trying GMane again after seeing that. One final thought: although Google is not the Python community, there are a lot of shared interests and overlap. Google sponsored and employed Guido for years. The most popular Tensorflow API is the Python API. Can't this relationship be used to ask Google to get more serious about the spam problem that originates in their domain, but that people in this discussion think they will hold against comp.lang.python? Alternately, how about giving Google Groups users killfiles and/or the same Spambayes filter tools that are already used on GMail inboxes? From breamoreboy at gmail.com Fri Feb 9 06:57:46 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Fri, 9 Feb 2018 11:57:46 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> Message-ID: On 09/02/18 11:23, John Ladasky wrote: > > I haven't visited GMane in a few years, but I found it difficult to navigate. In particular, I found searching to be cumbersome. Weren't the archives broken into 30-day blocks? > > I just tried GMane again two minutes ago. I can't confirm my recollections, but right now what I'm seeing is worse. If you follow this link right now... (http://gmane.org/find.php?list=comp.lang.python)... you get this: > > "Not all of Gmane is back yet - We're working hard to restore everything" > > And if you follow the link labeled "The latest news is at Gmane News" at the bottom of that page, it takes you here... (http://home.gmane.org/)... and the top blog post discussing site repairs is dated September 2016! > > I'm not too excited about trying GMane again after seeing that. > Please don't waste your time with the gmane website. Just point any (semi-)decent mail client like Thunderbird at news.gmane.org and you've access to hundreds of Python lists and thousands of other technical lists. I find the search facilities perfectly adequate. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From wolfgang.maier at biologie.uni-freiburg.de Fri Feb 9 06:59:38 2018 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Fri, 9 Feb 2018 12:59:38 +0100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> Message-ID: On 02/09/2018 12:23 PM, John Ladasky wrote: > On Friday, February 9, 2018 at 12:50:16 AM UTC-8, Tim Golden wrote: >> Gmane offers a newsgroup interface to the mailing list > > I haven't visited GMane in a few years, but I found it difficult to navigate. In particular, I found searching to be cumbersome. Weren't the archives broken into 30-day blocks? > > I just tried GMane again two minutes ago. I can't confirm my recollections, but right now what I'm seeing is worse. If you follow this link right now... (http://gmane.org/find.php?list=comp.lang.python)... you get this: > > "Not all of Gmane is back yet - We're working hard to restore everything" > > And if you follow the link labeled "The latest news is at Gmane News" at the bottom of that page, it takes you here... (http://home.gmane.org/)... and the top blog post discussing site repairs is dated September 2016! > > I'm not too excited about trying GMane again after seeing that. > You are certainly right about the gmane *Web Interface*. However, you can access their newsgroup interface through any newsgroup reader (including e.g. Thunderbird) and that's a really nice way of following things. From cl at isbd.net Fri Feb 9 07:45:29 2018 From: cl at isbd.net (Chris Green) Date: Fri, 9 Feb 2018 12:45:29 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro References: <4265150478@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> Message-ID: <9oh0le-r7g.ln1@esprimo.zbmc.eu> Mark Lawrence wrote: > On 09/02/18 11:23, John Ladasky wrote: > > > > I haven't visited GMane in a few years, but I found it difficult to navigate. > In particular, I found searching to be cumbersome. Weren't the archives > broken into 30-day blocks? > > > > I just tried GMane again two minutes ago. I can't confirm my recollections, > but right now what I'm seeing is worse. If you follow this link right now... > (http://gmane.org/find.php?list=comp.lang.python)... you get this: > > > > "Not all of Gmane is back yet - We're working hard to restore everything" > > > > And if you follow the link labeled "The latest news is at Gmane News" > at the bottom of that page, it takes you here... (http://home.gmane.org/)... > and the top blog post discussing site repairs is dated September 2016! > > > > > I'm not too excited about trying GMane again after seeing that. > > > > Please don't waste your time with the gmane website. Just point any > (semi-)decent mail client like Thunderbird at news.gmane.org and you've Surely you mean NNTP/Usenet client. > access to hundreds of Python lists and thousands of other technical > lists. I find the search facilities perfectly adequate. > -- Chris Green ? From steve+comp.lang.python at pearwood.info Fri Feb 9 09:07:36 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 9 Feb 2018 14:07:36 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro References: <4265150478@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> <9oh0le-r7g.ln1@esprimo.zbmc.eu> Message-ID: On Fri, 09 Feb 2018 12:45:29 +0000, Chris Green wrote: > Mark Lawrence wrote: [...] >> Please don't waste your time with the gmane website. Just point any >> (semi-)decent mail client like Thunderbird at news.gmane.org and you've > > Surely you mean NNTP/Usenet client. That's what he said :-) Thunderbird does both mail and news. Or at least it did, last time I checked. -- Steve From cl at isbd.net Fri Feb 9 10:37:04 2018 From: cl at isbd.net (Chris Green) Date: Fri, 9 Feb 2018 15:37:04 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro References: <4265150478@f38.n261.z1.binkp.net> <9oh0le-r7g.ln1@esprimo.zbmc.eu> Message-ID: <0qr0le-13j.ln1@esprimo.zbmc.eu> Steven D'Aprano wrote: > On Fri, 09 Feb 2018 12:45:29 +0000, Chris Green wrote: > > > Mark Lawrence wrote: > [...] > >> Please don't waste your time with the gmane website. Just point any > >> (semi-)decent mail client like Thunderbird at news.gmane.org and you've > > > > Surely you mean NNTP/Usenet client. > > That's what he said :-) > > Thunderbird does both mail and news. Or at least it did, last time I > checked. > He said ".....(semi-)decent mail client like Thunderbird...." That seems to imply (to me anyway) that one needs a mail client to read from news.gmane.org and that's just plain wrong. While *some* mail clients are also NNTP readers it's by no means universally true. -- Chris Green ? From jf_byrnes at comcast.net Fri Feb 9 10:40:02 2018 From: jf_byrnes at comcast.net (Jim) Date: Fri, 9 Feb 2018 09:40:02 -0600 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2a2bb132-f057-44f4-a1c0-47da0959b1c4@googlegroups.com> <9oh0le-r7g.ln1@esprimo.zbmc.eu> Message-ID: On 02/09/2018 08:07 AM, Steven D'Aprano wrote: > On Fri, 09 Feb 2018 12:45:29 +0000, Chris Green wrote: > >> Mark Lawrence wrote: > [...] >>> Please don't waste your time with the gmane website. Just point any >>> (semi-)decent mail client like Thunderbird at news.gmane.org and you've >> >> Surely you mean NNTP/Usenet client. > > That's what he said :-) > > Thunderbird does both mail and news. Or at least it did, last time I > checked. > > > It does, using it for this reply. Regards, Jim From mail at timgolden.me.uk Fri Feb 9 10:55:05 2018 From: mail at timgolden.me.uk (Tim Golden) Date: Fri, 9 Feb 2018 15:55:05 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: <0qr0le-13j.ln1@esprimo.zbmc.eu> References: <4265150478@f38.n261.z1.binkp.net> <9oh0le-r7g.ln1@esprimo.zbmc.eu> <0qr0le-13j.ln1@esprimo.zbmc.eu> Message-ID: On 09/02/2018 15:37, Chris Green wrote: > Steven D'Aprano wrote: >> On Fri, 09 Feb 2018 12:45:29 +0000, Chris Green wrote: >> >>> Mark Lawrence wrote: >> [...] >>>> Please don't waste your time with the gmane website. Just point any >>>> (semi-)decent mail client like Thunderbird at news.gmane.org and you've >>> >>> Surely you mean NNTP/Usenet client. >> >> That's what he said :-) >> >> Thunderbird does both mail and news. Or at least it did, last time I >> checked. >> > He said ".....(semi-)decent mail client like Thunderbird...." > > That seems to imply (to me anyway) that one needs a mail client to > read from news.gmane.org and that's just plain wrong. > > While *some* mail clients are also NNTP readers it's by no means > universally true. > Just to head this off at the pass: for the purpose of discussion, could we assume that this particular point doesn't really matter, please? Thanks TJG From dallasdisabilityattorney at gmail.com Fri Feb 9 13:35:15 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Fri, 9 Feb 2018 10:35:15 -0800 (PST) Subject: Extracting data from ython dictionary object In-Reply-To: References: <27be3bcf-a064-425f-9b11-edbead597db7@googlegroups.com> <8760761w13.fsf@handshake.de> Message-ID: On Friday, February 9, 2018 at 1:08:27 AM UTC-6, dieter wrote: > Stanley Denman writes: > > > I am new to Python. I am trying to extract text from the bookmarks in a PDF file that would provide the data for a Word template merge. I have gotten down to a string of text pulled out of the list object that I got from using PyPDF2 module. I am stuck on now to get the data out of the string that I need. I am calling it a string, but Python is recognizing as a dictionary object. > > > > Here is the string: > > > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > > > What a want is the following to end up as fields on my Word template merge: > > MedSourceFirstName: "John" > > MedSourceLastName: "Milani" > > MedSourceLastTreatment: "05/28/2014" > > > > If I use keys() on the dictionary I get this: > > ['/Title', '/Page', '/Type']I was hoping "Src" and Tmt Dt." would be treated as keys. Seems like the key/value pair of a dictionary would translate nicely to fieldname and fielddata for a Word document merge. Here is my code so far. > > A Python "dict" is a mapping of keys to values. Its "keys" method > gives you the keys (as you have used above). > The subscription syntax ("[]"; e.g. > "pdf_info['/Title']") allows you to access the value associated with > "". > > In your case, relevant information is coded inside the values themselves. > You will need to extract this information yourself. Python's "re" module > might be of help (see the "library reference", for details). Thanks for your response. Nice to know I am at least on the right path. Sounds like I am going to have to did in to Regex to get at the test I want. From dallasdisabilityattorney at gmail.com Fri Feb 9 13:40:35 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Fri, 9 Feb 2018 10:40:35 -0800 (PST) Subject: Extracting data from ython dictionary object (Posting On Python-List Prohibited) In-Reply-To: <501f9f9a-0455-4085-8f3b-538f5c9ecf2a@googlegroups.com> References: <27be3bcf-a064-425f-9b11-edbead597db7@googlegroups.com> <501f9f9a-0455-4085-8f3b-538f5c9ecf2a@googlegroups.com> Message-ID: On Friday, February 9, 2018 at 12:20:29 AM UTC-6, Lawrence D?Oliveiro wrote: > On Friday, February 9, 2018 at 6:04:48 PM UTC+13, Stanley Denman wrote: > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > > > What a want is the following to end up as fields on my Word template merge: > > MedSourceFirstName: "John" > > MedSourceLastName: "Milani" > > MedSourceLastTreatment: "05/28/2014" > > > > If I use keys() on the dictionary I get this: > > ['/Title', '/Page', '/Type']I was hoping "Src" and Tmt Dt." would be treated > > as keys. Seems like the key/value pair of a dictionary would translate > > nicely to fieldname and fielddata ... > > It would, except that?s not how the information is represented in the PDF file. Looks like what you want is all in the title string. So extracting it will require some string manipulation. Do all the title strings follow the same format? That should simplify the manipulations you need to do. Thanks you Lawrence for your response. Sounds like I am going to have to dig in to Regex to get at the test I want. From codewizard at gmail.com Fri Feb 9 14:10:49 2018 From: codewizard at gmail.com (codewizard at gmail.com) Date: Fri, 9 Feb 2018 11:10:49 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: <2802702295@f38.n261.z1.binkp.net> References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> Message-ID: <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: > [snip] > Those of us who do not use google-groups may not notice the loss > of the google groupies. I use GG to read comp.lang.python because of the following combination of factors. I would definitely be happier if there was another way to do this, so that I wouldn't have to wade through lots of spam. - I read it from both home and work. - I can't install any additional software at work. - There is no newsgroup reader software available at work. - GG tracks read/unread status of every post. This is shared between work and home through my Google account. - When I (very rarely) post from work, I can't do it by email, since outgoing email is blocked. Regards, Igor. From breamoreboy at gmail.com Fri Feb 9 14:32:25 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Fri, 9 Feb 2018 19:32:25 +0000 Subject: Extracting data from ython dictionary object In-Reply-To: References: <27be3bcf-a064-425f-9b11-edbead597db7@googlegroups.com> <8760761w13.fsf@handshake.de> Message-ID: On 09/02/18 18:35, Stanley Denman wrote: > On Friday, February 9, 2018 at 1:08:27 AM UTC-6, dieter wrote: >> Stanley Denman writes: >> >>> I am new to Python. I am trying to extract text from the bookmarks in a PDF file that would provide the data for a Word template merge. I have gotten down to a string of text pulled out of the list object that I got from using PyPDF2 module. I am stuck on now to get the data out of the string that I need. I am calling it a string, but Python is recognizing as a dictionary object. >>> >>> Here is the string: >>> >>> {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} >>> >>> What a want is the following to end up as fields on my Word template merge: >>> MedSourceFirstName: "John" >>> MedSourceLastName: "Milani" >>> MedSourceLastTreatment: "05/28/2014" >>> >>> If I use keys() on the dictionary I get this: >>> ['/Title', '/Page', '/Type']I was hoping "Src" and Tmt Dt." would be treated as keys. Seems like the key/value pair of a dictionary would translate nicely to fieldname and fielddata for a Word document merge. Here is my code so far. >> >> A Python "dict" is a mapping of keys to values. Its "keys" method >> gives you the keys (as you have used above). >> The subscription syntax ("[]"; e.g. >> "pdf_info['/Title']") allows you to access the value associated with >> "". >> >> In your case, relevant information is coded inside the values themselves. >> You will need to extract this information yourself. Python's "re" module >> might be of help (see the "library reference", for details). > > Thanks for your response. Nice to know I am at least on the right path. Sounds like I am going to have to did in to Regex to get at the test I want. > Maybe using string methods is simpler than a regex. >>> data = '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)' >>> bits = data.split(':') >>> bits ['1F', ' Progress Notes Src.', ' MILANI, JOHN C Tmt. Dt.', ' 05/12/2014 - 05/28/2014 (9 pages)'] >>> namebits = bits[2].split() >>> namebits ['MILANI,', 'JOHN', 'C', 'Tmt.', 'Dt.'] # I'll leave you to grab the names, and strip the comma from the last name. >>> start = bits[3].find('- ') >>> stop = bits[3].find('(') >>> date = bits[3][start + 2: stop].strip() >>> date '05/28/2014' Apologies for the variable names used, I'm sure that you can think of something better :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From cl at isbd.net Fri Feb 9 14:37:16 2018 From: cl at isbd.net (Chris Green) Date: Fri, 9 Feb 2018 19:37:16 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: codewizard at gmail.com wrote: > On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: > > [snip] > > Those of us who do not use google-groups may not notice the loss > > of the google groupies. > > I use GG to read comp.lang.python because of the following combination > of factors. I would definitely be happier if there was another way to > do this, so that I wouldn't have to wade through lots of spam. > > - I read it from both home and work. > > - I can't install any additional software at work. > > - There is no newsgroup reader software available at work. > > - GG tracks read/unread status of every post. This is shared > between work and home through my Google account. > > - When I (very rarely) post from work, I can't > do it by email, since outgoing email is blocked. > Alternative approach, what I do:- Run a text mode (but very capable and mouse aware) newsreader on my home system, read news locally using that. Use ssh (is that available at worK?) to connect from work to home and run the same newsreader in the same environment. If you can't even ssh from work then you can always use an 'ssh from the web' app from your wenb browser. The newsreader I use is tin by the way. -- Chris Green ? From codewizard at gmail.com Fri Feb 9 16:05:24 2018 From: codewizard at gmail.com (codewizard at gmail.com) Date: Fri, 9 Feb 2018 13:05:24 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: On Friday, February 9, 2018 at 2:48:17 PM UTC-5, Chris Green wrote: > codew... at gmail.com wrote: > > On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: > > > [snip] > > > Those of us who do not use google-groups may not notice the loss > > > of the google groupies. > > > > I use GG to read comp.lang.python because of the following combination > > of factors. I would definitely be happier if there was another way to > > do this, so that I wouldn't have to wade through lots of spam. > > > > - I read it from both home and work. > > > > - I can't install any additional software at work. > > > > - There is no newsgroup reader software available at work. > > > > - GG tracks read/unread status of every post. This is shared > > between work and home through my Google account. > > > > - When I (very rarely) post from work, I can't > > do it by email, since outgoing email is blocked. > > > Alternative approach, what I do:- > > Run a text mode (but very capable and mouse aware) newsreader on > my home system, read news locally using that. > > Use ssh (is that available at worK?) to connect from work to home > and run the same newsreader in the same environment. If you can't > even ssh from work then you can always use an 'ssh from the web' > app from your wenb browser. > > The newsreader I use is tin by the way. > > -- > Chris Green > ? ssh would be even a bigger problem here. As I mentioned, I can't even email from work to the outside! All web sites that provide remote connectivity tools are blocked. Regards, Igor. From rosuav at gmail.com Fri Feb 9 16:12:58 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 10 Feb 2018 08:12:58 +1100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: On Sat, Feb 10, 2018 at 8:05 AM, wrote: > On Friday, February 9, 2018 at 2:48:17 PM UTC-5, Chris Green wrote: >> codew... at gmail.com wrote: >> > On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: >> > > [snip] >> > > Those of us who do not use google-groups may not notice the loss >> > > of the google groupies. >> > >> > I use GG to read comp.lang.python because of the following combination >> > of factors. I would definitely be happier if there was another way to >> > do this, so that I wouldn't have to wade through lots of spam. >> > >> > - I read it from both home and work. >> > >> > - I can't install any additional software at work. >> > >> > - There is no newsgroup reader software available at work. >> > >> > - GG tracks read/unread status of every post. This is shared >> > between work and home through my Google account. >> > >> > - When I (very rarely) post from work, I can't >> > do it by email, since outgoing email is blocked. >> > >> Alternative approach, what I do:- >> >> Run a text mode (but very capable and mouse aware) newsreader on >> my home system, read news locally using that. >> >> Use ssh (is that available at worK?) to connect from work to home >> and run the same newsreader in the same environment. If you can't >> even ssh from work then you can always use an 'ssh from the web' >> app from your wenb browser. >> >> The newsreader I use is tin by the way. >> >> -- >> Chris Green >> ? > > ssh would be even a bigger problem here. As I mentioned, I can't even email > from work to the outside! All web sites that provide remote connectivity > tools are blocked. ... yet they're okay with you browsing newsgroups? I think you may have a Layer Eight conflict here. ChrisA From Richard at Damon-Family.org Fri Feb 9 17:03:27 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Fri, 9 Feb 2018 17:03:27 -0500 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: <95ed13b3-c8e6-cddc-b84e-e9a613e69a47@Damon-Family.org> On 2/9/18 4:12 PM, Chris Angelico wrote: > On Sat, Feb 10, 2018 at 8:05 AM, wrote: >> On Friday, February 9, 2018 at 2:48:17 PM UTC-5, Chris Green wrote: >>> codew... at gmail.com wrote: >>>> On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: >>>>> [snip] >>>>> Those of us who do not use google-groups may not notice the loss >>>>> of the google groupies. >>>> I use GG to read comp.lang.python because of the following combination >>>> of factors. I would definitely be happier if there was another way to >>>> do this, so that I wouldn't have to wade through lots of spam. >>>> >>>> - I read it from both home and work. >>>> >>>> - I can't install any additional software at work. >>>> >>>> - There is no newsgroup reader software available at work. >>>> >>>> - GG tracks read/unread status of every post. This is shared >>>> between work and home through my Google account. >>>> >>>> - When I (very rarely) post from work, I can't >>>> do it by email, since outgoing email is blocked. >>>> >>> Alternative approach, what I do:- >>> >>> Run a text mode (but very capable and mouse aware) newsreader on >>> my home system, read news locally using that. >>> >>> Use ssh (is that available at worK?) to connect from work to home >>> and run the same newsreader in the same environment. If you can't >>> even ssh from work then you can always use an 'ssh from the web' >>> app from your wenb browser. >>> >>> The newsreader I use is tin by the way. >>> >>> -- >>> Chris Green >>> ? >> ssh would be even a bigger problem here. As I mentioned, I can't even email >> from work to the outside! All web sites that provide remote connectivity >> tools are blocked. > ... yet they're okay with you browsing newsgroups? I think you may > have a Layer Eight conflict here. > > ChrisA If he can browse, he can get E-Mail with Gmail/Yahoo/AOL/etc, his normal mail ISP likely even supports some form of Web Mail Client. If they are blocking these, but not Google Groups, there is a major disconnect in the rules. -- Richard Damon From codewizard at gmail.com Fri Feb 9 18:19:28 2018 From: codewizard at gmail.com (codewizard at gmail.com) Date: Fri, 9 Feb 2018 15:19:28 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> <95ed13b3-c8e6-cddc-b84e-e9a613e69a47@Damon-Family.org> Message-ID: On Friday, February 9, 2018 at 5:03:45 PM UTC-5, Richard Damon wrote: > On 2/9/18 4:12 PM, Chris Angelico wrote: > > On Sat, Feb 10, 2018 at 8:05 AM, wrote: > >> On Friday, February 9, 2018 at 2:48:17 PM UTC-5, Chris Green wrote: > >>> codew... at gmail.com wrote: > >>>> On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: > >>>>> [snip] > >>>>> Those of us who do not use google-groups may not notice the loss > >>>>> of the google groupies. > >>>> I use GG to read comp.lang.python because of the following combination > >>>> of factors. I would definitely be happier if there was another way to > >>>> do this, so that I wouldn't have to wade through lots of spam. > >>>> > >>>> - I read it from both home and work. > >>>> > >>>> - I can't install any additional software at work. > >>>> > >>>> - There is no newsgroup reader software available at work. > >>>> > >>>> - GG tracks read/unread status of every post. This is shared > >>>> between work and home through my Google account. > >>>> > >>>> - When I (very rarely) post from work, I can't > >>>> do it by email, since outgoing email is blocked. > >>>> > >>> Alternative approach, what I do:- > >>> > >>> Run a text mode (but very capable and mouse aware) newsreader on > >>> my home system, read news locally using that. > >>> > >>> Use ssh (is that available at worK?) to connect from work to home > >>> and run the same newsreader in the same environment. If you can't > >>> even ssh from work then you can always use an 'ssh from the web' > >>> app from your wenb browser. > >>> > >>> The newsreader I use is tin by the way. > >>> > >>> -- > >>> Chris Green > >>> ? > >> ssh would be even a bigger problem here. As I mentioned, I can't even email > >> from work to the outside! All web sites that provide remote connectivity > >> tools are blocked. > > ... yet they're okay with you browsing newsgroups? I think you may > > have a Layer Eight conflict here. > > > > ChrisA > > If he can browse, he can get E-Mail with Gmail/Yahoo/AOL/etc, his normal > mail ISP likely even supports some form of Web Mail Client. > > If they are blocking these, but not Google Groups, there is a major > disconnect in the rules. > > -- > Richard Damon Disconnect or not, I can browse GG and post replies (e.g., this one). I have no access to gmail or any other email provider. Regards, Igor. From Richard at Damon-Family.org Fri Feb 9 18:44:57 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Fri, 9 Feb 2018 18:44:57 -0500 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> <95ed13b3-c8e6-cddc-b84e-e9a613e69a47@Damon-Family.org> Message-ID: On 2/9/18 6:19 PM, codewizard at gmail.com wrote: > On Friday, February 9, 2018 at 5:03:45 PM UTC-5, Richard Damon wrote: >> On 2/9/18 4:12 PM, Chris Angelico wrote: >>> On Sat, Feb 10, 2018 at 8:05 AM, wrote: >>>> On Friday, February 9, 2018 at 2:48:17 PM UTC-5, Chris Green wrote: >>>>> codew... at gmail.com wrote: >>>>>> On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: >>>>>>> [snip] >>>>>>> Those of us who do not use google-groups may not notice the loss >>>>>>> of the google groupies. >>>>>> I use GG to read comp.lang.python because of the following combination >>>>>> of factors. I would definitely be happier if there was another way to >>>>>> do this, so that I wouldn't have to wade through lots of spam. >>>>>> >>>>>> - I read it from both home and work. >>>>>> >>>>>> - I can't install any additional software at work. >>>>>> >>>>>> - There is no newsgroup reader software available at work. >>>>>> >>>>>> - GG tracks read/unread status of every post. This is shared >>>>>> between work and home through my Google account. >>>>>> >>>>>> - When I (very rarely) post from work, I can't >>>>>> do it by email, since outgoing email is blocked. >>>>>> >>>>> Alternative approach, what I do:- >>>>> >>>>> Run a text mode (but very capable and mouse aware) newsreader on >>>>> my home system, read news locally using that. >>>>> >>>>> Use ssh (is that available at worK?) to connect from work to home >>>>> and run the same newsreader in the same environment. If you can't >>>>> even ssh from work then you can always use an 'ssh from the web' >>>>> app from your wenb browser. >>>>> >>>>> The newsreader I use is tin by the way. >>>>> >>>>> -- >>>>> Chris Green >>>>> ? >>>> ssh would be even a bigger problem here. As I mentioned, I can't even email >>>> from work to the outside! All web sites that provide remote connectivity >>>> tools are blocked. >>> ... yet they're okay with you browsing newsgroups? I think you may >>> have a Layer Eight conflict here. >>> >>> ChrisA >> If he can browse, he can get E-Mail with Gmail/Yahoo/AOL/etc, his normal >> mail ISP likely even supports some form of Web Mail Client. >> >> If they are blocking these, but not Google Groups, there is a major >> disconnect in the rules. >> >> -- >> Richard Damon > Disconnect or not, I can browse GG and post replies (e.g., this one). > I have no access to gmail or any other email provider. > > Regards, > Igor. My guess then is that your access to google groups is also against your companies policies, they just haven't put an enforcement in their firewalls because it is too obscure for the IT department to have thought of. Based on the sort of broad blocking they are doing, I am a bit surprised they seem to be black listing rather than white listing the web. -- Richard Damon From codewizard at gmail.com Fri Feb 9 20:30:57 2018 From: codewizard at gmail.com (codewizard at gmail.com) Date: Fri, 9 Feb 2018 17:30:57 -0800 (PST) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> <95ed13b3-c8e6-cddc-b84e-e9a613e69a47@Damon-Family.org> Message-ID: <8e1a182a-1ec2-452f-9d6c-d92994dfcb10@googlegroups.com> On Friday, February 9, 2018 at 6:45:18 PM UTC-5, Richard Damon wrote: > On 2/9/18 6:19 PM, codew... at gmail.com wrote: > > On Friday, February 9, 2018 at 5:03:45 PM UTC-5, Richard Damon wrote: > >> On 2/9/18 4:12 PM, Chris Angelico wrote: > >>> On Sat, Feb 10, 2018 at 8:05 AM, wrote: > >>>> On Friday, February 9, 2018 at 2:48:17 PM UTC-5, Chris Green wrote: > >>>>> codew... at gmail.com wrote: > >>>>>> On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: > >>>>>>> [snip] > >>>>>>> Those of us who do not use google-groups may not notice the loss > >>>>>>> of the google groupies. > >>>>>> I use GG to read comp.lang.python because of the following combination > >>>>>> of factors. I would definitely be happier if there was another way to > >>>>>> do this, so that I wouldn't have to wade through lots of spam. > >>>>>> > >>>>>> - I read it from both home and work. > >>>>>> > >>>>>> - I can't install any additional software at work. > >>>>>> > >>>>>> - There is no newsgroup reader software available at work. > >>>>>> > >>>>>> - GG tracks read/unread status of every post. This is shared > >>>>>> between work and home through my Google account. > >>>>>> > >>>>>> - When I (very rarely) post from work, I can't > >>>>>> do it by email, since outgoing email is blocked. > >>>>>> > >>>>> Alternative approach, what I do:- > >>>>> > >>>>> Run a text mode (but very capable and mouse aware) newsreader on > >>>>> my home system, read news locally using that. > >>>>> > >>>>> Use ssh (is that available at worK?) to connect from work to home > >>>>> and run the same newsreader in the same environment. If you can't > >>>>> even ssh from work then you can always use an 'ssh from the web' > >>>>> app from your wenb browser. > >>>>> > >>>>> The newsreader I use is tin by the way. > >>>>> > >>>>> -- > >>>>> Chris Green > >>>>> ? > >>>> ssh would be even a bigger problem here. As I mentioned, I can't even email > >>>> from work to the outside! All web sites that provide remote connectivity > >>>> tools are blocked. > >>> ... yet they're okay with you browsing newsgroups? I think you may > >>> have a Layer Eight conflict here. > >>> > >>> ChrisA > >> If he can browse, he can get E-Mail with Gmail/Yahoo/AOL/etc, his normal > >> mail ISP likely even supports some form of Web Mail Client. > >> > >> If they are blocking these, but not Google Groups, there is a major > >> disconnect in the rules. > >> > >> -- > >> Richard Damon > > Disconnect or not, I can browse GG and post replies (e.g., this one). > > I have no access to gmail or any other email provider. > > > > Regards, > > Igor. > > My guess then is that your access to google groups is also against your > companies policies, they just haven't put an enforcement in their > firewalls because it is too obscure for the IT department to have > thought of. > > Based on the sort of broad blocking they are doing, I am a bit surprised > they seem to be black listing rather than white listing the web. > > -- > Richard Damon Regardless of what the policies are and how well they're enforced, if the mailing list is cut off from GG, I will stop reading it. Probably not a big loss to anyone else, but you might wonder how many others are in the same situation... Regards, Igor. From steve+comp.lang.python at pearwood.info Fri Feb 9 20:36:18 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 10 Feb 2018 01:36:18 +0000 (UTC) Subject: Incremental compression Message-ID: I want to compress a sequence of bytes one byte at a time. (I am already processing the bytes one byte at a time, for other reasons.) I don't particularly care *which* compression method is used, and in fact I'm not even interested in the compressed data itself, only its length. So I'm looking for something similar to this: count = 0 for b in stream: process(b) count += incremental_compressor.compressor(b) or some variation. Apart from bzip2, do I have any other options in the std lib? https://docs.python.org/3/library/bz2.html#incremental-de-compression -- Steve From drsalists at gmail.com Fri Feb 9 20:52:33 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 9 Feb 2018 17:52:33 -0800 Subject: Incremental compression In-Reply-To: References: Message-ID: Perhaps: import lzma lzc = lzma.LZMACompressor() out1 = lzc.compress(b"Some data\n") out2 = lzc.compress(b"Another piece of data\n") out3 = lzc.compress(b"Even more data\n") out4 = lzc.flush() # Concatenate all the partial results: result = b"".join([out1, out2, out3, out4]) ? lzma compresses harder than bzip2, but it's probably slower too. On Fri, Feb 9, 2018 at 5:36 PM, Steven D'Aprano wrote: > I want to compress a sequence of bytes one byte at a time. (I am already > processing the bytes one byte at a time, for other reasons.) I don't > particularly care *which* compression method is used, and in fact I'm not > even interested in the compressed data itself, only its length. So I'm > looking for something similar to this: > > count = 0 > for b in stream: > process(b) > count += incremental_compressor.compressor(b) > > > > or some variation. Apart from bzip2, do I have any other options in the > std lib? > > https://docs.python.org/3/library/bz2.html#incremental-de-compression > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list From steve+comp.lang.python at pearwood.info Fri Feb 9 23:44:54 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 10 Feb 2018 04:44:54 +0000 (UTC) Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> <95ed13b3-c8e6-cddc-b84e-e9a613e69a47@Damon-Family.org> <8e1a182a-1ec2-452f-9d6c-d92994dfcb10@googlegroups.com> Message-ID: On Fri, 09 Feb 2018 17:30:57 -0800, codewizard wrote: > Regardless of what the policies are and how well they're enforced, if > the mailing list is cut off from GG, I will stop reading it. There is *absolutely nothing* we can do about that. Google doesn't listen to us, and the suggestion that we can influence them because Guido van Rossum used to work for them many years ago is naive. We cannot influence Google's decision about the newsgroup; we cannot convince your boss to allow access to the newsgroup at work; and apparently we can't convince you that just because you will no longer be able to access it from work, doesn't mean you have to stop accessing it from home. Oh well, such is life. (Ironically, the vast bulk of spam on the newsgroup comes from Google Groups itself, so if this does happen, it will probably be a nett win for the community, even if it does negatively impact some individuals.) > Probably > not a big loss to anyone else, but you might wonder how many others are > in the same situation... There's nothing we can do about it, so there is no point wasting time wondering. -- Steve From steve+comp.lang.python at pearwood.info Sat Feb 10 00:56:22 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 10 Feb 2018 05:56:22 +0000 (UTC) Subject: Incremental compression References: Message-ID: On Fri, 09 Feb 2018 17:52:33 -0800, Dan Stromberg wrote: > Perhaps: > > import lzma > lzc = lzma.LZMACompressor() Ah, thanks for the suggestion! -- Steve From storchaka at gmail.com Sat Feb 10 06:20:13 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 10 Feb 2018 13:20:13 +0200 Subject: [RELEASED] Python 3.4.8 and Python 3.5.5 are now available In-Reply-To: <7b3f1a47-bdb1-8601-3a68-2b6c5b53b1ac@hastings.org> References: <7b3f1a47-bdb1-8601-3a68-2b6c5b53b1ac@hastings.org> Message-ID: 05.02.18 02:35, Larry Hastings ????: > On behalf of the Python development community, I'm happy to announce the > availability of Python 3.4.8 and Python 3.5.5. > > Both Python 3.4 and 3.5 are in "security fixes only" mode.? Both > versions only accept security fixes, not conventional bug fixes, and > both releases are source-only. > > > You can find Python 3.4.8 here: > > ?? https://www.python.org/downloads/release/python-348/ > > > And you can find Python 3.5.5 here: > > ?? https://www.python.org/downloads/release/python-355/ Online documentation for 3.5 is not updated yet. https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-5 From gheskett at shentel.net Sat Feb 10 09:21:28 2018 From: gheskett at shentel.net (Gene Heskett) Date: Sat, 10 Feb 2018 09:21:28 -0500 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <8e1a182a-1ec2-452f-9d6c-d92994dfcb10@googlegroups.com> Message-ID: <201802100921.29062.gheskett@shentel.net> On Friday 09 February 2018 23:44:54 Steven D'Aprano wrote: > On Fri, 09 Feb 2018 17:30:57 -0800, codewizard wrote: > > Regardless of what the policies are and how well they're enforced, > > if the mailing list is cut off from GG, I will stop reading it. > > There is *absolutely nothing* we can do about that. Google doesn't > listen to us, and the suggestion that we can influence them because > Guido van Rossum used to work for them many years ago is naive. > > We cannot influence Google's decision about the newsgroup; we cannot > convince your boss to allow access to the newsgroup at work; and > apparently we can't convince you that just because you will no longer > be able to access it from work, doesn't mean you have to stop > accessing it from home. Oh well, such is life. > > (Ironically, the vast bulk of spam on the newsgroup comes from Google > Groups itself, so if this does happen, it will probably be a nett win > for the community, even if it does negatively impact some > individuals.) > > > Probably > > not a big loss to anyone else, but you might wonder how many others > > are in the same situation... > > There's nothing we can do about it, so there is no point wasting time > wondering. > The list I see here is much much cleaner, as anything from google groups has been diverted to /dev/null for a year or more. We don't need the blather that comes out of googlegroups. If google is unhappy and cuts it off, methinks they should run a broom through their own real estate first. > > -- > Steve -- Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From acharbly at gmail.com Sat Feb 10 09:43:35 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Sat, 10 Feb 2018 20:13:35 +0530 Subject: Jython info In-Reply-To: References: Message-ID: Hello team, Is it right to ask Jython related query here? If so kindly check my q here otherwise sorry for posting here... Can we access swing objects/events of any applications via Jython? My present code Downloaded Jython 2.5 Import sys Sys.path.append (my java. Jar) Thanks Prahallad From cwr at seckford.org Sat Feb 10 09:57:40 2018 From: cwr at seckford.org (C W Rose) Date: Sat, 10 Feb 2018 14:57:40 +0000 Subject: Spam levels. Message-ID: <4sd3le-ct4.ln1@tanner.seckford.org> I've been reading a limited range of Usenet groups since the late 1980s, and until the recent problems in comp.lang.python had never bothered with any sort of filtering; it's easier just to ignore people. However, the sheer volume of spam in comp.lang.python finally defeated me, so I set up a filter in my leafnode NNTP server, which pulls from eternal-september and gmane (read-only). The results for the last few weeks are below: Jan 20 15:34:39 comp.lang.python: 50 articles fetched, 71 killed Jan 21 14:21:30 comp.lang.python: 25 articles fetched, 18 killed Jan 26 13:04:40 comp.lang.python: 85 articles fetched, 276 killed Jan 28 22:58:42 comp.lang.python: 83 articles fetched, 184 killed Jan 29 16:07:11 comp.lang.python: 41 articles fetched, 52 killed Jan 30 16:57:03 comp.lang.python: 57 articles fetched, 56 killed Jan 31 16:52:01 comp.lang.python: 39 articles fetched, 87 killed Feb 1 16:02:49 comp.lang.python: 39 articles fetched, 73 killed Feb 2 14:39:46 comp.lang.python: 43 articles fetched, 57 killed Feb 3 18:17:55 comp.lang.python: 19 articles fetched, 108 killed Feb 4 15:28:03 comp.lang.python: 36 articles fetched, 122 killed Feb 5 15:26:08 comp.lang.python: 45 articles fetched, 79 killed Feb 6 16:29:40 comp.lang.python: 68 articles fetched, 93 killed Feb 7 16:43:26 comp.lang.python: 32 articles fetched, 118 killed Feb 8 16:01:14 comp.lang.python: 41 articles fetched, 100 killed Feb 9 17:06:09 comp.lang.python: 47 articles fetched, 201 killed giving the totals: Kept: 750 articles Killed: 1695 articles Until the last few days the spammers were using a fixed format, but recently I've had to play whack-a-mole with filtering, so the Killed total is probably underestimated by ten or twenty posts. No other groups (in the limited set which I read) have the problem, and I don't understand why the spammers neither spam a range of groups, nor change their adddresses more frequently. It may be that destroying comp.lang.python is their actual objective. Either way, a depressing state of affairs. Will -- "As democracy is perfected, the office of the president represents, more and more closely, the inner soul of the people. On some great and glorious day, the plain folks of the land will reach their heart's desire at last, and the White House will be occupied by a downright fool and a complete narcissistic moron." -- H. L. Mencken. From porton at narod.ru Sat Feb 10 09:58:09 2018 From: porton at narod.ru (Victor Porton) Date: Sat, 10 Feb 2018 16:58:09 +0200 Subject: $srcdir and $datadir References: <854lmrf5f1.fsf@benfinney.id.au> Message-ID: First, I've already solved my problem using setuptools and pkg_resources.resource_stream() and an environment variable to specify the path to data files. Ben Finney wrote: > Victor Porton writes: > >> In GNU software written in C $srcdir and $datadir are accessible to C >> code through generated config.h file. > > For what purpose? I want my program to work both when it is installed (using $datadir) and when it is not yet installed (using $srcdir). > Given that the source may not be at that location after the program is > compiled ? especially, after the program is moved to a different machine > ? what meaning does ?$srcdir? have when the program is running? In GNU system there is support of storing a value (such as source directory) into the program itself. If it is moved to a different machine, $srcdir remains the same. > What ?data directory? is specified by ?$datadir?, and why is it assumed > there is exactly one? It may be exactly one, but contain subdirectories. >> What is the right way to config directories for a Python program? > > We'll need to know what those concepts mean, to be able to discuss the > equivalent (if any) in a Python environment. -- Victor Porton - http://portonvictor.org From drsalists at gmail.com Sat Feb 10 11:20:28 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sat, 10 Feb 2018 08:20:28 -0800 Subject: Jython info In-Reply-To: References: Message-ID: On Sat, Feb 10, 2018 at 6:43 AM, Prahallad Achar wrote: > Hello team, > Is it right to ask Jython related query here? If so kindly check my q here > otherwise sorry for posting here... I believe it can be done: http://lmgtfy.com/?q=jython+swing By the way, Jython 2.5 is pretty old. From acharbly at gmail.com Sat Feb 10 11:38:47 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Sat, 10 Feb 2018 22:08:47 +0530 Subject: Jython info In-Reply-To: References: Message-ID: Thanks for the kind response. Yes. I shall upgrade the same. am trying to access GUI action elements from Jython to automate the Swing desktop application On Sat, Feb 10, 2018 at 9:50 PM, Dan Stromberg wrote: > On Sat, Feb 10, 2018 at 6:43 AM, Prahallad Achar > wrote: > > Hello team, > > Is it right to ask Jython related query here? If so kindly check my q > here > > otherwise sorry for posting here... > > I believe it can be done: > http://lmgtfy.com/?q=jython+swing > > By the way, Jython 2.5 is pretty old. > From none at invalid.com Sat Feb 10 12:12:55 2018 From: none at invalid.com (mm0fmf) Date: Sat, 10 Feb 2018 17:12:55 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: On 09/02/2018 21:05, codewizard at gmail.com wrote: > On Friday, February 9, 2018 at 2:48:17 PM UTC-5, Chris Green wrote: >> codew... at gmail.com wrote: >>> On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich wrote: >>>> [snip] >>>> Those of us who do not use google-groups may not notice the loss >>>> of the google groupies. >>> >>> I use GG to read comp.lang.python because of the following combination >>> of factors. I would definitely be happier if there was another way to >>> do this, so that I wouldn't have to wade through lots of spam. >>> >>> - I read it from both home and work. >>> >>> - I can't install any additional software at work. >>> >>> - There is no newsgroup reader software available at work. >>> >>> - GG tracks read/unread status of every post. This is shared >>> between work and home through my Google account. >>> >>> - When I (very rarely) post from work, I can't >>> do it by email, since outgoing email is blocked. >>> >> Alternative approach, what I do:- >> >> Run a text mode (but very capable and mouse aware) newsreader on >> my home system, read news locally using that. >> >> Use ssh (is that available at worK?) to connect from work to home >> and run the same newsreader in the same environment. If you can't >> even ssh from work then you can always use an 'ssh from the web' >> app from your wenb browser. >> >> The newsreader I use is tin by the way. >> >> -- >> Chris Green >> ? > > ssh would be even a bigger problem here. As I mentioned, I can't even email > from work to the outside! All web sites that provide remote connectivity > tools are blocked. > > Regards, > Igor. > If they take such restrictive practices then maybe should consider *just* doing your job when at work and leave the browsing to your own time? From michael.stemper at gmail.com Sat Feb 10 16:39:21 2018 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Sat, 10 Feb 2018 15:39:21 -0600 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: On 2018-02-09 13:37, Chris Green wrote: > Alternative approach, what I do:- > > Run a text mode (but very capable and mouse aware) newsreader on > my home system, read news locally using that. > > Use ssh (is that available at worK?) to connect from work to home Your ISP provides that capability? I'm surprised. I'm with Charter and the only external IP address is for the cable modem (I think). "icanhazip" shows the same one, no matter which computer I use. -- Michael F. Stemper Psalm 94:3-6 From cl at isbd.net Sat Feb 10 16:52:11 2018 From: cl at isbd.net (Chris Green) Date: Sat, 10 Feb 2018 21:52:11 +0000 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: Michael F. Stemper wrote: > On 2018-02-09 13:37, Chris Green wrote: > > > Alternative approach, what I do:- > > > > Run a text mode (but very capable and mouse aware) newsreader on > > my home system, read news locally using that. > > > > Use ssh (is that available at worK?) to connect from work to home > > Your ISP provides that capability? I'm surprised. I'm with Charter and > the only external IP address is for the cable modem (I think). > "icanhazip" shows the same one, no matter which computer I use. > You use the same external IP address, just use the right port number (which you set up your router to allow in). -- Chris Green ? From rosuav at gmail.com Sat Feb 10 17:40:40 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 11 Feb 2018 09:40:40 +1100 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: On Sun, Feb 11, 2018 at 8:52 AM, Chris Green wrote: > Michael F. Stemper wrote: >> On 2018-02-09 13:37, Chris Green wrote: >> >> > Alternative approach, what I do:- >> > >> > Run a text mode (but very capable and mouse aware) newsreader on >> > my home system, read news locally using that. >> > >> > Use ssh (is that available at worK?) to connect from work to home >> >> Your ISP provides that capability? I'm surprised. I'm with Charter and >> the only external IP address is for the cable modem (I think). >> "icanhazip" shows the same one, no matter which computer I use. >> > You use the same external IP address, just use the right port number > (which you set up your router to allow in). > Exactly. The easiest way is to set up port forwarding in your cable modem. Any device worth using should have at least SOME measure of port forwarding. Of course, if your ISP offers an IPv6 netblock, it's easy. ChrisA From ben+python at benfinney.id.au Sat Feb 10 18:49:09 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 11 Feb 2018 10:49:09 +1100 Subject: $srcdir and $datadir References: <854lmrf5f1.fsf@benfinney.id.au> Message-ID: <851shse77u.fsf@benfinney.id.au> Victor Porton writes: > I want my program to work both when it is installed (using $datadir) > and when it is not yet installed (using $srcdir). Given your descriptions of both those, I don't see how it can reliably work; the program will expect directories that may not exist. Why hard-code them at all, then? So, I am going to have to ignore the ?$srcdir and $datadir? specifics, and address the stated intent. The current convention in Python build systems is the Setuptools library. You will be familiar with the ?setup.py? top-level script for co-ordinating Setuptools actions on a code base. The installation tool ?pip? works with Setuptools data. Use it to install your distribution from PyPI for production use; use it to install your code base from local directories for testing; use it with special options to ?install? your code in a development mode while you work on it. -- \ ?Always code as if the guy who ends up maintaining your code | `\ will be a violent psychopath who knows where you live.? ?John | _o__) F. Woods | Ben Finney From larry at hastings.org Sat Feb 10 19:17:32 2018 From: larry at hastings.org (Larry Hastings) Date: Sat, 10 Feb 2018 16:17:32 -0800 Subject: [RELEASED] Python 3.4.8 and Python 3.5.5 are now available In-Reply-To: References: <7b3f1a47-bdb1-8601-3a68-2b6c5b53b1ac@hastings.org> Message-ID: Actually, it was updated on the server, but somehow the old version was sticking around in the CDN cache.? I "purged" it and it's fine now.? Weird that it would linger this long! Cheers, //arry/ On 02/10/2018 03:20 AM, Serhiy Storchaka wrote: > 05.02.18 02:35, Larry Hastings ????: >> On behalf of the Python development community, I'm happy to announce >> the availability of Python 3.4.8 and Python 3.5.5. >> >> Both Python 3.4 and 3.5 are in "security fixes only" mode.? Both >> versions only accept security fixes, not conventional bug fixes, and >> both releases are source-only. >> >> >> You can find Python 3.4.8 here: >> >> ??? https://www.python.org/downloads/release/python-348/ >> >> >> And you can find Python 3.5.5 here: >> >> ??? https://www.python.org/downloads/release/python-355/ > > Online documentation for 3.5 is not updated yet. > > > https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-5 > From larry at hastings.org Sat Feb 10 21:15:09 2018 From: larry at hastings.org (Larry Hastings) Date: Sat, 10 Feb 2018 18:15:09 -0800 Subject: [RELEASED] Python 3.4.8 and Python 3.5.5 are now available In-Reply-To: References: <7b3f1a47-bdb1-8601-3a68-2b6c5b53b1ac@hastings.org> Message-ID: <81687a94-0ed0-1edb-c3f4-1ac6e6577d29@hastings.org> Actually, it was updated on the server, but somehow the old version was sticking around in the CDN cache.? I "purged" it and it's fine now.? Weird that it would linger this long! Cheers, //arry/ On 02/10/2018 03:20 AM, Serhiy Storchaka wrote: > 05.02.18 02:35, Larry Hastings ????: >> On behalf of the Python development community, I'm happy to announce >> the availability of Python 3.4.8 and Python 3.5.5. >> >> Both Python 3.4 and 3.5 are in "security fixes only" mode.? Both >> versions only accept security fixes, not conventional bug fixes, and >> both releases are source-only. >> >> >> You can find Python 3.4.8 here: >> >> ??? https://www.python.org/downloads/release/python-348/ >> >> >> And you can find Python 3.5.5 here: >> >> ??? https://www.python.org/downloads/release/python-355/ > > Online documentation for 3.5 is not updated yet. > > > https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-5 > From alister.ware at ntlworld.com Sun Feb 11 06:35:45 2018 From: alister.ware at ntlworld.com (alister) Date: Sun, 11 Feb 2018 11:35:45 GMT Subject: This newsgroup (comp.lang.python) may soon be blocked by Google References: <1259629769@f38.n261.z1.binkp.net> <626072225@f38.n261.z1.binkp.net> Message-ID: On Sat, 10 Feb 2018 17:12:54 +1200, mm0fmf wrote: > On 09/02/2018 21:05, codewizard at gmail.com wrote: >> On Friday, February 9, 2018 at 2:48:17 PM UTC-5, Chris Green wrote: >>> codew... at gmail.com wrote: >>>> On Saturday, February 3, 2018 at 7:15:16 PM UTC-5, pyotr filipivich >>>> wrote: >>>>> [snip] >>>>> Those of us who do not use google-groups may not notice the >>>>> loss >>>>> of the google groupies. >>>> >>>> I use GG to read comp.lang.python because of the following >>>> combination of factors. I would definitely be happier if there was >>>> another way to do this, so that I wouldn't have to wade through lots >>>> of spam. >>>> >>>> - I read it from both home and work. >>>> >>>> - I can't install any additional software at work. >>>> >>>> - There is no newsgroup reader software available at work. >>>> >>>> - GG tracks read/unread status of every post. This is shared >>>> between work and home through my Google account. >>>> >>>> - When I (very rarely) post from work, I can't >>>> do it by email, since outgoing email is blocked. >>>> >>> Alternative approach, what I do:- >>> >>> Run a text mode (but very capable and mouse aware) newsreader on >>> my home system, read news locally using that. >>> >>> Use ssh (is that available at worK?) to connect from work to home >>> and run the same newsreader in the same environment. If you can't >>> even ssh from work then you can always use an 'ssh from the web' >>> app from your wenb browser. >>> >>> The newsreader I use is tin by the way. >>> >>> -- >>> Chris Green ?? >> >> ssh would be even a bigger problem here. As I mentioned, I can't even >> email from work to the outside! All web sites that provide remote >> connectivity tools are blocked. >> >> Regards, >> Igor. >> > If they take such restrictive practices then maybe should consider > *just* doing your job when at work and leave the browsing to your own > time? I would also suggest that you consider the above statement ha snot been made out of malice but is trying to protect you. if the restrictions are as strict as you say and you get seen you will probably loose your job. -- Xerox does it again and again and again and ... From porton at narod.ru Sun Feb 11 07:30:24 2018 From: porton at narod.ru (Victor Porton) Date: Sun, 11 Feb 2018 14:30:24 +0200 Subject: What is more Pythonic: subclass or adding functionality to base class? Message-ID: The following is a real code fragment: ########################################################## class PredicateParser(object, metaclass=ABCMeta): """ Parses a given predicate (which may participate in several relationships) of a given RDF node. """ def __init__(self, predicate): self.predicate = predicate @abstractmethod def parse(self, parse_context, graph, node): pass ########################################################## Now I need to add new field on_error (taking one of three enumerated values) to some objects of this class. What is more pythonic? 1. Create its subclass PredicateParserWithError and add the additional field on_error to this class. 2. Add on_error field to the base class, setting it to None by default, if the class's user does not need this field. -- Victor Porton - http://portonvictor.org From darcy at VybeNetworks.com Sun Feb 11 07:55:04 2018 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Sun, 11 Feb 2018 06:55:04 -0600 Subject: What is more Pythonic: subclass or adding functionality to base class? In-Reply-To: References: Message-ID: On 02/11/18 06:30, Victor Porton wrote: > What is more pythonic? > > 1. Create its subclass PredicateParserWithError and add the additional field > on_error to this class. > > 2. Add on_error field to the base class, setting it to None by default, if > the class's user does not need this field. Personally I would go with #1. It just seems cleaner. Not sure what the Python gods would do. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From arj.python at gmail.com Sun Feb 11 08:05:34 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 11 Feb 2018 17:05:34 +0400 Subject: python desktop app making experience In-Reply-To: References: Message-ID: Hi all, Just can you provide your own experience of developing and shipping a desktop app. Did you here about such a task before etc, until years after story. Abdur-Rahmaan Janhangeer https://abdurrahmaanjanhangeer.wordpress.com From p.f.moore at gmail.com Sun Feb 11 08:41:36 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 11 Feb 2018 13:41:36 +0000 Subject: What is more Pythonic: subclass or adding functionality to base class? In-Reply-To: References: Message-ID: On 11 February 2018 at 12:55, D'Arcy Cain wrote: > On 02/11/18 06:30, Victor Porton wrote: >> What is more pythonic? >> >> 1. Create its subclass PredicateParserWithError and add the additional field >> on_error to this class. >> >> 2. Add on_error field to the base class, setting it to None by default, if >> the class's user does not need this field. > > Personally I would go with #1. It just seems cleaner. Not sure what > the Python gods would do. I don't know about "Python Gods" or even what's more Pythonic, but it might be worth commenting that if this were Java, adding a new field to the base class would need every file that used the class to be recompiled. So there's a tendency to subclass just to minimise the impact of a change like this. In Python, that's not necessary, as classes are created at runtime, and there's no compile-time layout that code needs to depend on, so if you add an extra field to a class, only those parts of your codebase that need the new field have to care. Of course, if this is a public API, backward compatibility and versioning of the API become issues, that may affect your decision. Paul From arj.python at gmail.com Sun Feb 11 11:24:04 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 11 Feb 2018 20:24:04 +0400 Subject: python desktop app making experience In-Reply-To: References: Message-ID: my bad *hear Abdur-Rahmaan Janhangeer https://abdurrahmaanjanhangeer.wordpress.com On 11 Feb 2018 17:05, "Abdur-Rahmaan Janhangeer" wrote: > Hi all, > > Just can you provide your own experience of developing and shipping a > desktop app. Did you here about such a task before etc, until years after > story. > > Abdur-Rahmaan Janhangeer > https://abdurrahmaanjanhangeer.wordpress.com > From martin.schoon at gmail.com Sun Feb 11 16:19:26 2018 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 11 Feb 2018 21:19:26 GMT Subject: Adding colormaps? References: Message-ID: Den 2017-01-23 skrev Albert-Jan Roskam : > (sorry for top-posting) > I does not appear to be possible in matplolibrc (1). But you can > use matplotlib.cm.register_cmap to register new cmaps (2) such > as these (3). > > (Note: I did not try this) > > (1)http://matplotlib.org/1.4.0/users/customizing.html > (2)http://matplotlib.org/api/cm_api.html > (3)https://github.com/BIDS/colormap/blob/master/colormaps.py More than a year later I think I can advice on the best workaround: Use virtualenv and you can work with newer versions of Python packages than those available in your Linux distro of base Python installation. I am currently learning to use jupyter notebook, pandas and other nice packages in much more recent versions than available in my Linux distro. I hope I can copy this onto my computer at work -- running another Linux distro where virtualenv3 does not seem to be available. /Martin From sjsumitj at gmail.com Mon Feb 12 04:19:47 2018 From: sjsumitj at gmail.com (Sum J) Date: Mon, 12 Feb 2018 14:49:47 +0530 Subject: [Python-Dev] How to set/update value in a xml file using requests in python In-Reply-To: References: Message-ID: Thanks Dan for your response. I have updated the data to be sent in xml format and now I am able to update the required fields using put request. data= ''' web-proxy.xx.com 8080 on ''' On Thu, Feb 8, 2018 at 10:15 PM, Dan Stromberg wrote: > This is more relevant to python-list than python-dev. I've added > python-list to the To header. > > Gmail doesn't appear to allow setting a reply-to for a single message, > so I've not set that; please, when replying, drop python-dev from the > to: header. > > You'll likely want to set up some kind of REST endpoint to allow > updating your xml file. Imagine if anyone could change anyone else's > xml files - it'd be pretty chaotic. > > Perhaps flask would be appropriate? > http://flask.pocoo.org/docs/0.12/quickstart/ > > You likely will want a PUT or a POST HTTP verb: > https://stackoverflow.com/questions/107390/whats-the- > difference-between-a-post-and-a-put-http-request > > HTH. > > On Thu, Feb 8, 2018 at 12:38 AM, Sum J wrote: > > My xml file is located in local network: > > > > http://192.168.43.109/DevMgmt/NetAppsDyn.xml > > > > Below is a part content of above xml I want to update : > > > > > > > > > > > > > > off > > > > > > > > > > I want to set the value for 'ResourceUI' and 'Port' field in above xml. > > > > I have used below code : > > > > import requests > > data = { > > 'ResourceURI':'web-proxy.xxx.yy.com', > > 'Port':8080 > > } > > > > URL = 'http://192.168.75.165/DevMgmt/NetAppsDyn.xml' > > > > # content type > > head = {'Content-type': 'text/xml'} > > # sending get request > > gr= requests.get(url=URL) > > print gr > > > > # sending put request > > r = requests.put(url=URL, data=data,headers=head) > > print r.status_code > > # extracting response text > > output_xml = r.text > > print("The op xml is:%s" % output_xml) > > > > Issue : The fields are not getting updated in xml using put request. I am > > able to see the response for get (request) , but for put request it is > > throwing errror code : 301 , resource has been moved permanently. > > > > > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > https://mail.python.org/mailman/options/python-dev/drsalists%40gmail.com > > > From pahome.chen at mirlab.org Mon Feb 12 04:40:13 2018 From: pahome.chen at mirlab.org (lampahome) Date: Mon, 12 Feb 2018 17:40:13 +0800 Subject: abspath returns different results when py_compile input path is different in Python? Message-ID: I want to know abspath of python script followed by steps below. 1. *built it to byte code by py_compile.* 2. *execute it to check abspath.* But I got *2 results* when I execute it.I found the *results based on the path of script* followed by py_compile. Here is my script test.py : import osimport inspect print os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe()))) Build it with py_compile, then got 2 results when I enter *different path of test.py*: *enter the folder and compile with only script name.* [~]cd /usr/local/bin/[/usr/local/bin/]python -m py_compile test.py[/usr/local/bin/]cd ~[~]python /usr/local/bin/test.pyc/home/UserXX *In other folder and compile with absolute script name.* [~]python -m py_compile /usr/local/bin/test.py[~]python /usr/local/bin/test.pyc/usr/local/bin how come got 2 different results? From eryksun at gmail.com Mon Feb 12 06:27:51 2018 From: eryksun at gmail.com (eryk sun) Date: Mon, 12 Feb 2018 11:27:51 +0000 Subject: abspath returns different results when py_compile input path is different in Python? In-Reply-To: References: Message-ID: On Mon, Feb 12, 2018 at 9:40 AM, lampahome wrote: > I want to know abspath of python script followed by steps below. > > 1. *built it to byte code by py_compile.* > 2. *execute it to check abspath.* > > But I got *2 results* when I execute it.I found the *results based on the > path of script* followed by py_compile. > > Here is my script test.py : > > import os > import inspect > print os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe()))) > > Build it with py_compile, then got 2 results when I enter *different path > of test.py*: > > *enter the folder and compile with only script name.* > > [~] cd /usr/local/bin/ > [/usr/local/bin/] python -m py_compile test.py > [/usr/local/bin/] cd ~ > [~] python /usr/local/bin/test.pyc > /home/UserXX > > *In other folder and compile with absolute script name.* > > [~] python -m py_compile /usr/local/bin/test.py > [~] python /usr/local/bin/test.pyc > /usr/local/bin A code object has a co_filename attribute for use in creating tracebacks. This path isn't necessarily fully qualified. Here are a couple of examples using the built-in compile() function: >>> compile('42', 'test.py', 'exec').co_filename 'test.py' >>> compile('42', '/usr/local/bin/test.py', 'exec').co_filename '/usr/local/bin/test.py' py_compile.compile() does not normalize the filename. From pedronac at gmail.com Mon Feb 12 12:54:29 2018 From: pedronac at gmail.com (Pedro Crescencio) Date: Mon, 12 Feb 2018 18:54:29 +0100 Subject: Can't Uninstall ! In-Reply-To: <000001d3a41b$4cf40890$e6dc19b0$@gmail.com> References: <000001d3a41b$4cf40890$e6dc19b0$@gmail.com> Message-ID: <000801d3a42a$883235d0$9896a170$@gmail.com> Hello, I tried to uninstall Python, but it is not possible. Even when message say it is uninstalled : The software is still on applications menu: How do I do to uninstall it ? I have the same problem with booth python versions (3.6.4 and 3.7.0b1) and the Sourcetree of Atlassian. Best regards, Pedro From michael.stemper at gmail.com Mon Feb 12 19:06:51 2018 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 12 Feb 2018 18:06:51 -0600 Subject: Can't Uninstall ! In-Reply-To: References: <000001d3a41b$4cf40890$e6dc19b0$@gmail.com> <000801d3a42a$883235d0$9896a170$@gmail.com> Message-ID: On 2018-02-12 11:54, Pedro Crescencio wrote: > Hello, > I tried to uninstall Python, but it is not possible. > > Even when message say it is uninstalled : > The software is still on applications menu: > How do I do to uninstall it ? > > I have the same problem with booth python versions (3.6.4 and 3.7.0b1) and > the Sourcetree of Atlassian. A few questions: 1. Why do you want to uninstall it? What are you trying to accomplish by uninstalling it? 2. What operating system are you using? 3. How did you try to uninstall it? 4. What messages did you get when you tried to uninstall it? 5. How did you install two different 3.x versions? -- Michael F. Stemper Indians scattered on dawn's highway bleeding; Ghosts crowd the young child's fragile eggshell mind. From pahome.chen at mirlab.org Mon Feb 12 20:48:16 2018 From: pahome.chen at mirlab.org (lampahome) Date: Tue, 13 Feb 2018 09:48:16 +0800 Subject: abspath returns different results when py_compile input path is different in Python? In-Reply-To: References: Message-ID: so py_compile.compile() doesn't normalize the filename, but it will pass the input filename to co_filename? 2018-02-12 19:27 GMT+08:00 eryk sun : > On Mon, Feb 12, 2018 at 9:40 AM, lampahome wrote: > > I want to know abspath of python script followed by steps below. > > > > 1. *built it to byte code by py_compile.* > > 2. *execute it to check abspath.* > > > > But I got *2 results* when I execute it.I found the *results based on the > > path of script* followed by py_compile. > > > > Here is my script test.py : > > > > import os > > import inspect > > print os.path.dirname(os.path.abspath(inspect.getfile( > inspect.currentframe()))) > > > > Build it with py_compile, then got 2 results when I enter *different path > > of test.py*: > > > > *enter the folder and compile with only script name.* > > > > [~] cd /usr/local/bin/ > > [/usr/local/bin/] python -m py_compile test.py > > [/usr/local/bin/] cd ~ > > [~] python /usr/local/bin/test.pyc > > /home/UserXX > > > > *In other folder and compile with absolute script name.* > > > > [~] python -m py_compile /usr/local/bin/test.py > > [~] python /usr/local/bin/test.pyc > > /usr/local/bin > > A code object has a co_filename attribute for use in creating > tracebacks. This path isn't necessarily fully qualified. Here are a > couple of examples using the built-in compile() function: > > >>> compile('42', 'test.py', 'exec').co_filename > 'test.py' > >>> compile('42', '/usr/local/bin/test.py', 'exec').co_filename > '/usr/local/bin/test.py' > > py_compile.compile() does not normalize the filename. > From cwr at seckford.org Tue Feb 13 07:49:26 2018 From: cwr at seckford.org (C W Rose) Date: Tue, 13 Feb 2018 12:49:26 +0000 Subject: Respam levels. Message-ID: Having run a check for straightforward spam, I now find that there's a site editing and reposting non-spam posts. An example of the changed headers follows: Original post headers: > > From cwr at seckford.org Sun Feb 11 23:23:22 2018 > Path: eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail > Message-ID: <4sd3le-ct4.ln1 at tanner.seckford.org> > From: C W Rose > Newsgroups: comp.lang.python > Subject: Spam levels. > Date: Sat, 10 Feb 2018 14:57:40 +0000 > Lines: 49 > Organization: None > Injection-Info: reader02.eternal-september.org; posting-host="208e6410f66c7dd789dea67159b51bb7"; > logging-data="6406"; mail-complaints-to="abuse at eternal-september.org"; posting-account="U2FsdGVkX18su+yG7dqCpnqChqoPIY0JNXRwgNWzvcs=" > User-Agent: tin/2.0.1-20111224 ("Achenvoir") (UNIX) (Linux/3.10.79-gentoo-1 (i686)) > Cancel-Lock: sha1:H7zOidR0ivbNjqiWUB6DSoMMYu8= > Xref: tanner.seckford.org comp.lang.python:139461 > Repost headers: > > From C Sun Feb 11 23:22:54 2018 > Path: eternal-september.org!reader02.eternal-september.org!feeder.eternal-september.org!aioe.org!news.bbs.geek.nz!.POSTED.agency.bbs.geek.nz!not-for-mail > Message-ID: <847574370 at f38.n261.z1.binkp.net> > From: C W Rose (C W Rose) > Newsgroups: comp.lang.python > Subject: Spam levels. > Date: Sat, 10 Feb 2018 14:57:40 +1200 > Organization: fsxNet Usenet Gateway | bbs.nz/#fsxNet > Mime-Version: 1.0 > Content-Type: text/plain; charset=us-ascii > Content-Transfer-Encoding: 7bit > Injection-Info: news.bbs.geek.nz; posting-host="M6YmRdZYyc42DJk0lNlt/X4dpP4dzvceBNabSmESN3E"; > logging-data="24378"; mail-complaints-to="abuse at news.bbs.geek.nz" > User-Agent: VSoup v1.2.9.47Beta [95/NT] > X-MailConverter: SoupGate-Win32 v1.05 > X-Comment-To: All > Xref: tanner.seckford.org comp.lang.python:139479 Looking at my current comp.lang.python news spool, 182 out of 683 messages have been reposted by news.bbs.geek.nz; I haven't checked if they are all duplicates. The reformatting of the body of message is to date trivial, just a reduction in line length. This may be the result of a misconfigured spam filter, or an actual spam attack; anyway, I've now filtered news.bbs.geek.nz from my feed. Will -- "When we look around us, this is what we find, The hope that springs eternal, springs right up your behind." -- Ian Dury From dallasdisabilityattorney at gmail.com Tue Feb 13 08:11:20 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Tue, 13 Feb 2018 05:11:20 -0800 (PST) Subject: Regex on a Dictionary Message-ID: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> I am trying to performance a regex on a "string" of text that python isinstance is telling me is a dictionary. When I run the code I get the following error: {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} Traceback (most recent call last): File "C:\Users\stand\Desktop\PythonSublimeText.py", line 9, in x=MyRegex.findall(MyDict) TypeError: expected string or bytes-like object Here is the "string" of code I am working with: {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} I want to grab the name "MILANI, JOHN C" and the last date "-mm/dd/yyyy" as a pair such that if I have X numbers of string like the above I will end out with N pairs of values (name and date)/ Here is my code: import PyPDF2,re pdfFileObj=open('x.pdf','rb') pdfReader=PyPDF2.PdfFileReader(pdfFileObj) Result=pdfReader.getOutlines() MyDict=(Result[-1][0]) print(MyDict) print(isinstance(MyDict,dict)) MyRegex=re.compile(r"MILANI,") x=MyRegex.findall(MyDict) print(x) Thanks in advance for any help. From michael.stemper at gmail.com Tue Feb 13 08:28:18 2018 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 13 Feb 2018 07:28:18 -0600 Subject: This newsgroup (comp.lang.python) may soon be blocked by Google Gro In-Reply-To: References: <4265150478@f38.n261.z1.binkp.net> <2802702295@f38.n261.z1.binkp.net> <71e7537f-b747-4f35-82ae-5152efcb955f@googlegroups.com> Message-ID: On 2018-02-10 16:40, Chris Angelico wrote: > On Sun, Feb 11, 2018 at 8:52 AM, Chris Green wrote: >> Michael F. Stemper wrote: >>> On 2018-02-09 13:37, Chris Green wrote: >>>> Use ssh (is that available at worK?) to connect from work to home >>> >>> Your ISP provides that capability? I'm surprised. I'm with Charter and >>> the only external IP address is for the cable modem (I think). >>> "icanhazip" shows the same one, no matter which computer I use. >>> >> You use the same external IP address, just use the right port number >> (which you set up your router to allow in). > > Exactly. The easiest way is to set up port forwarding in your cable > modem. Any device worth using should have at least SOME measure of > port forwarding. Of course, if your ISP offers an IPv6 netblock, it's > easy. Ah, the light dawns over Marblehead! When I set up my router, I saw the "port forwarding" page, but had no idea what it was for. Thanks for the explanation. I will now cease off-topic posting. -- Michael F. Stemper This post contains greater than 95% post-consumer bytes by weight. From tkadm30 at yandex.com Tue Feb 13 08:29:06 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 13 Feb 2018 08:29:06 -0500 Subject: Regex on a Dictionary In-Reply-To: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> Message-ID: Hi Stanley, Le 2018-02-13 ? 08:11, Stanley Denman a ?crit?: > x=MyRegex.findall(MyDict) How about: x = [MyRegex.findall(item) for item in MyDict] Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From breamoreboy at gmail.com Tue Feb 13 08:38:11 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Tue, 13 Feb 2018 13:38:11 +0000 Subject: Respam levels. In-Reply-To: References: Message-ID: On 13/02/18 12:49, C W Rose via Python-list wrote: > > Having run a check for straightforward spam, I now find that there's a site > editing and reposting non-spam posts. An example of the changed headers > follows: > > Original post headers: >> >> From cwr at seckford.org Sun Feb 11 23:23:22 2018 >> Path: eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail >> Message-ID: <4sd3le-ct4.ln1 at tanner.seckford.org> >> From: C W Rose >> Newsgroups: comp.lang.python >> Subject: Spam levels. >> Date: Sat, 10 Feb 2018 14:57:40 +0000 >> Lines: 49 >> Organization: None >> Injection-Info: reader02.eternal-september.org; posting-host="208e6410f66c7dd789dea67159b51bb7"; >> logging-data="6406"; mail-complaints-to="abuse at eternal-september.org"; posting-account="U2FsdGVkX18su+yG7dqCpnqChqoPIY0JNXRwgNWzvcs=" >> User-Agent: tin/2.0.1-20111224 ("Achenvoir") (UNIX) (Linux/3.10.79-gentoo-1 (i686)) >> Cancel-Lock: sha1:H7zOidR0ivbNjqiWUB6DSoMMYu8= >> Xref: tanner.seckford.org comp.lang.python:139461 >> > > Repost headers: >> >> From C Sun Feb 11 23:22:54 2018 >> Path: eternal-september.org!reader02.eternal-september.org!feeder.eternal-september.org!aioe.org!news.bbs.geek.nz!.POSTED.agency.bbs.geek.nz!not-for-mail >> Message-ID: <847574370 at f38.n261.z1.binkp.net> >> From: C W Rose (C W Rose) >> Newsgroups: comp.lang.python >> Subject: Spam levels. >> Date: Sat, 10 Feb 2018 14:57:40 +1200 >> Organization: fsxNet Usenet Gateway | bbs.nz/#fsxNet >> Mime-Version: 1.0 >> Content-Type: text/plain; charset=us-ascii >> Content-Transfer-Encoding: 7bit >> Injection-Info: news.bbs.geek.nz; posting-host="M6YmRdZYyc42DJk0lNlt/X4dpP4dzvceBNabSmESN3E"; >> logging-data="24378"; mail-complaints-to="abuse at news.bbs.geek.nz" >> User-Agent: VSoup v1.2.9.47Beta [95/NT] >> X-MailConverter: SoupGate-Win32 v1.05 >> X-Comment-To: All >> Xref: tanner.seckford.org comp.lang.python:139479 > > Looking at my current comp.lang.python news spool, 182 out of 683 messages > have been reposted by news.bbs.geek.nz; I haven't checked if they are all > duplicates. The reformatting of the body of message is to date trivial, > just a reduction in line length. > > This may be the result of a misconfigured spam filter, or an actual spam > attack; anyway, I've now filtered news.bbs.geek.nz from my feed. > > Will > IIRC the same source for the "nospam" stuff of some months ago which I believe was purely accidental. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rhodri at kynesim.co.uk Tue Feb 13 08:42:08 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 13 Feb 2018 13:42:08 +0000 Subject: Regex on a Dictionary In-Reply-To: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> Message-ID: On 13/02/18 13:11, Stanley Denman wrote: > I am trying to performance a regex on a "string" of text that python isinstance is telling me is a dictionary. When I run the code I get the following error: > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > Traceback (most recent call last): > File "C:\Users\stand\Desktop\PythonSublimeText.py", line 9, in > x=MyRegex.findall(MyDict) > TypeError: expected string or bytes-like object > > Here is the "string" of code I am working with: > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > I want to grab the name "MILANI, JOHN C" and the last date "-mm/dd/yyyy" as a pair such that if I have X numbers of string like the above I will end out with N pairs of values (name and date)/ Here is my code: > > import PyPDF2,re > pdfFileObj=open('x.pdf','rb') > pdfReader=PyPDF2.PdfFileReader(pdfFileObj) > Result=pdfReader.getOutlines() > MyDict=(Result[-1][0]) > print(MyDict) > print(isinstance(MyDict,dict)) > MyRegex=re.compile(r"MILANI,") > x=MyRegex.findall(MyDict) > print(x) As the error message says, re.findall() expects a string. A dictionary is in no sense a string, so passing it in whole like that won't work. If you know that the name will always show up in the title field, you can pass just the title: x = MyRegex.findall(MyDict['/Title']) Otherwise you will have to loop through all the entries in the dictionary: for entry in MyDict.values(): x = MyRegex.findall(entry) # ...and do something with x I rather suspect you are going to find that the titles aren't in a very systematic format, though. -- Rhodri James *-* Kynesim Ltd From breamoreboy at gmail.com Tue Feb 13 08:53:04 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Tue, 13 Feb 2018 13:53:04 +0000 Subject: Regex on a Dictionary In-Reply-To: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> Message-ID: On 13/02/18 13:11, Stanley Denman wrote: > I am trying to performance a regex on a "string" of text that python isinstance is telling me is a dictionary. When I run the code I get the following error: > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > Traceback (most recent call last): > File "C:\Users\stand\Desktop\PythonSublimeText.py", line 9, in > x=MyRegex.findall(MyDict) > TypeError: expected string or bytes-like object > > Here is the "string" of code I am working with: Please call it a dictionary as in the subject line, quite clearly it is not a string in any way, shape or form. > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > I want to grab the name "MILANI, JOHN C" and the last date "-mm/dd/yyyy" as a pair such that if I have X numbers of string like the above I will end out with N pairs of values (name and date)/ Here is my code: > > import PyPDF2,re > pdfFileObj=open('x.pdf','rb') > pdfReader=PyPDF2.PdfFileReader(pdfFileObj) > Result=pdfReader.getOutlines() > MyDict=(Result[-1][0]) > print(MyDict) > print(isinstance(MyDict,dict)) > MyRegex=re.compile(r"MILANI,") > x=MyRegex.findall(MyDict) > print(x) > > Thanks in advance for any help. > Was the string methods solution that I gave a week or so ago so bad that you still think that you need a regex to solve this? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From alister.ware at ntlworld.com Tue Feb 13 10:14:16 2018 From: alister.ware at ntlworld.com (alister) Date: Tue, 13 Feb 2018 15:14:16 GMT Subject: Regex on a Dictionary References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> Message-ID: On Tue, 13 Feb 2018 13:42:08 +0000, Rhodri James wrote: > On 13/02/18 13:11, Stanley Denman wrote: >> I am trying to performance a regex on a "string" of text that python >> isinstance is telling me is a dictionary. When I run the code I get >> the following error: >> >> {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: >> 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), >> '/Type': '/FitB'} >> >> Traceback (most recent call last): >> File "C:\Users\stand\Desktop\PythonSublimeText.py", line 9, in >> >> x=MyRegex.findall(MyDict) >> TypeError: expected string or bytes-like object >> >> Here is the "string" of code I am working with: >> >> {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: >> 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), >> '/Type': '/FitB'} >> >> I want to grab the name "MILANI, JOHN C" and the last date >> "-mm/dd/yyyy" as a pair such that if I have X numbers of string like >> the above I will end out with N pairs of values (name and date)/ Here >> is my code: >> >> import PyPDF2,re pdfFileObj=open('x.pdf','rb') >> pdfReader=PyPDF2.PdfFileReader(pdfFileObj) >> Result=pdfReader.getOutlines() >> MyDict=(Result[-1][0]) >> print(MyDict) >> print(isinstance(MyDict,dict)) >> MyRegex=re.compile(r"MILANI,") >> x=MyRegex.findall(MyDict) >> print(x) > > As the error message says, re.findall() expects a string. A dictionary > is in no sense a string, so passing it in whole like that won't work. > If you know that the name will always show up in the title field, you > can pass just the title: > > x = MyRegex.findall(MyDict['/Title']) > > Otherwise you will have to loop through all the entries in the > dictionary: > > for entry in MyDict.values(): > x = MyRegex.findall(entry) # ...and do something with x > > I rather suspect you are going to find that the titles aren't in a very > systematic format, though. for what purpose are you trying to run this regex anyway? it is almost certainly the wrong approach for your task -- Larkinson's Law: All laws are basically false. From alister.ware at ntlworld.com Tue Feb 13 10:16:09 2018 From: alister.ware at ntlworld.com (alister) Date: Tue, 13 Feb 2018 15:16:09 GMT Subject: Respam levels. References: Message-ID: On Tue, 13 Feb 2018 12:49:26 +0000, C W Rose wrote: > Having run a check for straightforward spam, I now find that there's a > site editing and reposting non-spam posts. An example of the changed > headers follows: > > Original post headers: >> >> From cwr at seckford.org Sun Feb 11 23:23:22 2018 Path: >> eternal-september.org!reader02.eternal-september.org!.POSTED!not-for- mail >> Message-ID: <4sd3le-ct4.ln1 at tanner.seckford.org> >> From: C W Rose >> Newsgroups: comp.lang.python Subject: Spam levels. >> Date: Sat, 10 Feb 2018 14:57:40 +0000 Lines: 49 Organization: None >> Injection-Info: reader02.eternal-september.org; >> posting-host="208e6410f66c7dd789dea67159b51bb7"; >> logging-data="6406"; mail-complaints-to="abuse at eternal- september.org"; >> posting-account="U2FsdGVkX18su+yG7dqCpnqChqoPIY0JNXRwgNWzvcs=" >> User-Agent: tin/2.0.1-20111224 ("Achenvoir") (UNIX) >> (Linux/3.10.79-gentoo-1 (i686)) >> Cancel-Lock: sha1:H7zOidR0ivbNjqiWUB6DSoMMYu8= >> Xref: tanner.seckford.org comp.lang.python:139461 >> >> > Repost headers: >> >> From C Sun Feb 11 23:22:54 2018 Path: >> eternal-september.org!reader02.eternal-september.org!feeder.eternal- september.org!aioe.org!news.bbs.geek.nz!.POSTED.agency.bbs.geek.nz!not- for-mail >> Message-ID: <847574370 at f38.n261.z1.binkp.net> >> From: C W Rose (C W Rose) >> Newsgroups: comp.lang.python Subject: Spam levels. >> Date: Sat, 10 Feb 2018 14:57:40 +1200 Organization: fsxNet Usenet >> Gateway | bbs.nz/#fsxNet Mime-Version: 1.0 Content-Type: text/plain; >> charset=us-ascii Content-Transfer-Encoding: 7bit Injection-Info: >> news.bbs.geek.nz; >> posting-host="M6YmRdZYyc42DJk0lNlt/X4dpP4dzvceBNabSmESN3E"; >> logging-data="24378"; mail-complaints-to="abuse at news.bbs.geek.nz" >> User-Agent: VSoup v1.2.9.47Beta [95/NT] >> X-MailConverter: SoupGate-Win32 v1.05 X-Comment-To: All Xref: >> tanner.seckford.org comp.lang.python:139479 > > Looking at my current comp.lang.python news spool, 182 out of 683 > messages have been reposted by news.bbs.geek.nz; I haven't checked if > they are all duplicates. The reformatting of the body of message is to > date trivial, just a reduction in line length. > > This may be the result of a misconfigured spam filter, or an actual spam > attack; anyway, I've now filtered news.bbs.geek.nz from my feed. > > Will many thanks for the effort /dev/null on my laptop is now almost full with this junk :-) -- Let's do it. -- Gary Gilmore, to his firing squad From pedronac at gmail.com Tue Feb 13 10:30:36 2018 From: pedronac at gmail.com (Pedro Crescencio) Date: Tue, 13 Feb 2018 16:30:36 +0100 Subject: Can't Uninstall ! In-Reply-To: References: <000001d3a41b$4cf40890$e6dc19b0$@gmail.com> <000801d3a42a$883235d0$9896a170$@gmail.com> Message-ID: <002701d3a4df$98c0c9b0$ca425d10$@gmail.com> Hello Michael, Here my answers 1. Why do you want to uninstall it? What are you trying to accomplish by uninstalling it? -> I installed Python to test a software. Now I do not need python any more. 2. What operating system are you using? -> Windows 10 Familly 64b 3. How did you try to uninstall it? Using Windows standard uninstall procedure https://www.windowscentral.com/how-to-uninstall-apps-windows-10 4. What messages did you get when you tried to uninstall it? The message says : "Uninstalled". But I still have the icon over the App list. 5. How did you install two different 3.x versions? I tried to reinstall to do a proper uninstall. I tried with the latest version that might fix some bugs. -----Message d'origine----- De?: Python-list [mailto:python-list-bounces+pedronac=gmail.com at python.org] De la part de Michael F. Stemper Envoy??: mardi 13 f?vrier 2018 01:07 ??: python-list at python.org Objet?: Re: Can't Uninstall ! On 2018-02-12 11:54, Pedro Crescencio wrote: > Hello, > I tried to uninstall Python, but it is not possible. > > Even when message say it is uninstalled : > The software is still on applications menu: > How do I do to uninstall it ? > > I have the same problem with booth python versions (3.6.4 and 3.7.0b1) > and the Sourcetree of Atlassian. A few questions: 1. Why do you want to uninstall it? What are you trying to accomplish by uninstalling it? 2. What operating system are you using? 3. How did you try to uninstall it? 4. What messages did you get when you tried to uninstall it? 5. How did you install two different 3.x versions? -- Michael F. Stemper Indians scattered on dawn's highway bleeding; Ghosts crowd the young child's fragile eggshell mind. -- https://mail.python.org/mailman/listinfo/python-list From jasonhihn at gmail.com Tue Feb 13 11:02:38 2018 From: jasonhihn at gmail.com (Jason) Date: Tue, 13 Feb 2018 08:02:38 -0800 (PST) Subject: Defer, ensure library is loaded Message-ID: <2a0adcd7-4331-4ae1-bc6e-d0a5df86b69f@googlegroups.com> I have a variety of scripts that import some large libraries, and rather than create a million little scripts with specific imports, I'd like to so something like psycopg2 = ensure_imported (psycopg2) This way, regardless of invocation I can know psycopg2 is loaded, if it hasn't already been loaded. If I just do a normal import 95% of the time I'll be waiting time with a meaningless import. Can I do that somehow? From rosuav at gmail.com Tue Feb 13 11:08:07 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 14 Feb 2018 03:08:07 +1100 Subject: Defer, ensure library is loaded In-Reply-To: <2a0adcd7-4331-4ae1-bc6e-d0a5df86b69f@googlegroups.com> References: <2a0adcd7-4331-4ae1-bc6e-d0a5df86b69f@googlegroups.com> Message-ID: On Wed, Feb 14, 2018 at 3:02 AM, Jason wrote: > I have a variety of scripts that import some large libraries, and rather than create a million little scripts with specific imports, I'd like to so something like > > psycopg2 = ensure_imported (psycopg2) > > This way, regardless of invocation I can know psycopg2 is loaded, if it hasn't already been loaded. If I just do a normal import 95% of the time I'll be waiting time with a meaningless import. > > > Can I do that somehow? Yep! It's written like this: import psycopg2 If it's already been imported, Python will go fetch it straight from the cache, so it's fast. (The cache is in "sys.modules", if you're curious. "import sys" to have a look at it.) ChrisA From steve+comp.lang.python at pearwood.info Tue Feb 13 12:06:05 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 13 Feb 2018 17:06:05 +0000 (UTC) Subject: Regex on a Dictionary References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> Message-ID: On Tue, 13 Feb 2018 05:11:20 -0800, Stanley Denman wrote: > I am trying to performance a regex on a "string" of text that python > isinstance is telling me is a dictionary. Please believe Python when it tells you that something is a dictionary. Trust me, the interpreter knows. It doesn't matter how much you want something to be a string, if it is not a string, it isn't a string. -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 13 12:16:20 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 13 Feb 2018 17:16:20 +0000 (UTC) Subject: Regex on a Dictionary References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> Message-ID: On Tue, 13 Feb 2018 13:53:04 +0000, Mark Lawrence wrote: > Was the string methods solution that I gave a week or so ago so bad that > you still think that you need a regex to solve this? Sometimes regexes are needed, but often Jamie Zawinski is right: Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems. Using the nuclear-powered bulldozer of regular expressions to crack the peanut of a simple fixed-string matching problem is rarely a good idea. -- Steve From israel at ravnalaska.net Tue Feb 13 12:28:47 2018 From: israel at ravnalaska.net (Israel Brewster) Date: Tue, 13 Feb 2018 08:28:47 -0900 Subject: Packaging uwsgi flask app for non-programmers? In-Reply-To: <605B7863-F542-4D54-A45A-1AD5284807F7@ravnalaska.net> References: <605B7863-F542-4D54-A45A-1AD5284807F7@ravnalaska.net> Message-ID: > > On Feb 6, 2018, at 12:12 PM, Israel Brewster wrote: > > I have been working on writing an Alexa skill which, as part of it, requires a local web server on the end users machine - the Alexa skill sends commands to this server, which runs them on the local machine. I wrote this local server in Flask, and run it using uwsgi, using a command like: "uwsgi serverconfig.ini". > > The problem is that in order for this to work, the end user must: > > 1) Install python 3.6 (or thereabouts) > 2) Install a number of python modules, and > 3) run a command line (from the appropriate directory) > > Not terribly difficult, but when I think of my target audience (Alexa users), I could easily see even these steps being "too complicated". I was looking at pyinstaller to create a simple double-click application, but it appears that pyinstaller needs a python script as the "base" for the application, whereas my "base" is uwsgi. Also, I do need to leave a config file accessible for the end user to be able to edit. Is there a way to use pyinstaller in this scenario, or perhaps some other option that might work better to package things up? So at the moment, since there have been no suggestions for packaging, I'm getting by with a bash script that: a) Makes sure python 3 is installed, prompting the user to install it if not b) Makes sure pip and virtualenv are installed, and installs them if needed c) Sets up a virtualenv in the distribution directory d) Installs all needed modules in the virtualenv - this step requires that dev tools are installed, a separate install. e) modifies the configuration files to match the user and directory, and f) Installs a launchd script to run the uwsgi application This actually seems to work fairly well, and by giving the script a .command extension, which automatically gets associated with terminal under OS X, the end user can simply double-click setup.command without having to go into terminal themselves. The main stumbling block then is the install of python3 - the user still has to manually download and install it in addition to my code, which I'd prefer to avoid - having to install my code separate from the Alexa skill is already an annoyance. As such, I'm considering three possible solutions: 1) Make some sort of installer package that includes the python3 installer 2) Somehow automate the download and install of Python3, or 3) re-write my code to be python 2 compatible (since python 2 is included with the OS) If anyone has any suggestions on how I could accomplish 1 or 2, I'd appreciate it. Thanks! ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- > > ----------------------------------------------- > Israel Brewster > Systems Analyst II > Ravn Alaska > 5245 Airport Industrial Rd > Fairbanks, AK 99709 > (907) 450-7293 > ----------------------------------------------- > > > > > -- > https://mail.python.org/mailman/listinfo/python-list From malaclypse2 at gmail.com Tue Feb 13 13:01:22 2018 From: malaclypse2 at gmail.com (Jerry Hill) Date: Tue, 13 Feb 2018 13:01:22 -0500 Subject: Can't Uninstall ! In-Reply-To: <002701d3a4df$98c0c9b0$ca425d10$@gmail.com> References: <000001d3a41b$4cf40890$e6dc19b0$@gmail.com> <000801d3a42a$883235d0$9896a170$@gmail.com> <002701d3a4df$98c0c9b0$ca425d10$@gmail.com> Message-ID: On Tue, Feb 13, 2018 at 10:30 AM, Pedro Crescencio wrote: 3. How did you try to uninstall it? > Using Windows standard uninstall procedure > https://www.windowscentral.com/how-to-uninstall-apps-windows-10 > ?That page describes two different ways of uninstalling programs (from the start screen, or from the Apps & Features control panel). I don't know if it matters, but it might help someone to know which one you used.? > 4. What messages did you get when you tried to uninstall it? > The message says : "Uninstalled". But I still have the icon over the App > list. > ?You say you have "the" icon.? ?What icon is that? A normal python install should have a bunch of different stuff on the start menu -- on the Windows 7 installation ?I have in front of me, I have a "Python 3.6" folder, with four different links nested inside - "IDLE (Python 3.6 32-bit)", "Python 3.6 (32-bit)", "Python 3.6 Manuals (32-bit)", and "Python 3.6 Module Docs (32-bit)". What do you have? ?When you installed python, do you remember if you installed it for "All Users" or "Just Me"?? -- Jerry From dallasdisabilityattorney at gmail.com Tue Feb 13 13:08:35 2018 From: dallasdisabilityattorney at gmail.com (Stanley Denman) Date: Tue, 13 Feb 2018 10:08:35 -0800 (PST) Subject: Regex on a Dictionary In-Reply-To: References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> Message-ID: <423885d6-54b7-4e10-8e8a-dde2681154e0@googlegroups.com> On Tuesday, February 13, 2018 at 9:41:14 AM UTC-6, Mark Lawrence wrote: > On 13/02/18 13:11, Stanley Denman wrote: > > I am trying to performance a regex on a "string" of text that python isinstance is telling me is a dictionary. When I run the code I get the following error: > > > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > > > Traceback (most recent call last): > > File "C:\Users\stand\Desktop\PythonSublimeText.py", line 9, in > > x=MyRegex.findall(MyDict) > > TypeError: expected string or bytes-like object > > > > Here is the "string" of code I am working with: > > Please call it a dictionary as in the subject line, quite clearly it is > not a string in any way, shape or form. > > > > > {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} > > > > I want to grab the name "MILANI, JOHN C" and the last date "-mm/dd/yyyy" as a pair such that if I have X numbers of string like the above I will end out with N pairs of values (name and date)/ Here is my code: > > > > import PyPDF2,re > > pdfFileObj=open('x.pdf','rb') > > pdfReader=PyPDF2.PdfFileReader(pdfFileObj) > > Result=pdfReader.getOutlines() > > MyDict=(Result[-1][0]) > > print(MyDict) > > print(isinstance(MyDict,dict)) > > MyRegex=re.compile(r"MILANI,") > > x=MyRegex.findall(MyDict) > > print(x) > > > > Thanks in advance for any help. > > > > Was the string methods solution that I gave a week or so ago so bad that > you still think that you need a regex to solve this? > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence My Apology Mark. You took the time to give me the basis of a non-regex solution and I had not taken the time to fully review your answer.Did not understand it at first blush, but I think now I do. From pedronac at gmail.com Tue Feb 13 13:16:32 2018 From: pedronac at gmail.com (Pedro Crescencio) Date: Tue, 13 Feb 2018 19:16:32 +0100 Subject: Can't Uninstall ! In-Reply-To: References: <000001d3a41b$4cf40890$e6dc19b0$@gmail.com> <000801d3a42a$883235d0$9896a170$@gmail.com> <002701d3a4df$98c0c9b0$ca425d10$@gmail.com> Message-ID: <003601d3a4f6$c6c90810$545b1830$@gmail.com> RESOLVED To answer Jerry, I found solution. Point 3. First try: uninstall using Windows 10 interface to uninstall Like this https://malwaretips.com/blogs/wp-content/uploads/2015/07/Uninstall-Application-from-Windows-10.jpg Second try: right click over the program bouton on Start Menu 1st option of this link https://www.windowscentral.com/how-to-uninstall-apps-windows-10 It send to "Windows 7" like interface menu : http://www.uninstalltips.net/wp-content/uploads/2014/04/uninstall-program.png The second works! Can not explain why, but it does. It's good enough. Thanks -----Message d'origine----- De : Python-list [mailto:python-list-bounces+pedronac=gmail.com at python.org] De la part de Jerry Hill Envoy? : mardi 13 f?vrier 2018 19:01 ? : python-list (General) Objet : Re: Can't Uninstall ! On Tue, Feb 13, 2018 at 10:30 AM, Pedro Crescencio wrote: 3. How did you try to uninstall it? > Using Windows standard uninstall procedure > https://www.windowscentral.com/how-to-uninstall-apps-windows-10 > ?That page describes two different ways of uninstalling programs (from the start screen, or from the Apps & Features control panel). I don't know if it matters, but it might help someone to know which one you used.? > 4. What messages did you get when you tried to uninstall it? > The message says : "Uninstalled". But I still have the icon over the > App list. > ?You say you have "the" icon.? ?What icon is that? A normal python install should have a bunch of different stuff on the start menu -- on the Windows 7 installation ?I have in front of me, I have a "Python 3.6" folder, with four different links nested inside - "IDLE (Python 3.6 32-bit)", "Python 3.6 (32-bit)", "Python 3.6 Manuals (32-bit)", and "Python 3.6 Module Docs (32-bit)". What do you have? ?When you installed python, do you remember if you installed it for "All Users" or "Just Me"?? -- Jerry -- https://mail.python.org/mailman/listinfo/python-list From breamoreboy at gmail.com Tue Feb 13 13:59:16 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Tue, 13 Feb 2018 18:59:16 +0000 Subject: Regex on a Dictionary In-Reply-To: <423885d6-54b7-4e10-8e8a-dde2681154e0@googlegroups.com> References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> <423885d6-54b7-4e10-8e8a-dde2681154e0@googlegroups.com> Message-ID: On 13/02/18 18:08, Stanley Denman wrote: > On Tuesday, February 13, 2018 at 9:41:14 AM UTC-6, Mark Lawrence wrote: >> On 13/02/18 13:11, Stanley Denman wrote: >>> I am trying to performance a regex on a "string" of text that python isinstance is telling me is a dictionary. When I run the code I get the following error: >>> >>> {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} >>> >>> Traceback (most recent call last): >>> File "C:\Users\stand\Desktop\PythonSublimeText.py", line 9, in >>> x=MyRegex.findall(MyDict) >>> TypeError: expected string or bytes-like object >>> >>> Here is the "string" of code I am working with: >> >> Please call it a dictionary as in the subject line, quite clearly it is >> not a string in any way, shape or form. >> >>> >>> {'/Title': '1F: Progress Notes Src.: MILANI, JOHN C Tmt. Dt.: 05/12/2014 - 05/28/2014 (9 pages)', '/Page': IndirectObject(465, 0), '/Type': '/FitB'} >>> >>> I want to grab the name "MILANI, JOHN C" and the last date "-mm/dd/yyyy" as a pair such that if I have X numbers of string like the above I will end out with N pairs of values (name and date)/ Here is my code: >>> >>> import PyPDF2,re >>> pdfFileObj=open('x.pdf','rb') >>> pdfReader=PyPDF2.PdfFileReader(pdfFileObj) >>> Result=pdfReader.getOutlines() >>> MyDict=(Result[-1][0]) >>> print(MyDict) >>> print(isinstance(MyDict,dict)) >>> MyRegex=re.compile(r"MILANI,") >>> x=MyRegex.findall(MyDict) >>> print(x) >>> >>> Thanks in advance for any help. >>> >> >> Was the string methods solution that I gave a week or so ago so bad that >> you still think that you need a regex to solve this? >> >> -- >> My fellow Pythonistas, ask not what our language can do for you, ask >> what you can do for our language. >> >> Mark Lawrence > > My Apology Mark. You took the time to give me the basis of a non-regex solution and I had not taken the time to fully review your answer.Did not understand it at first blush, but I think now I do. > Accepted :) IIRC you might need a small tweak or two but certainly the foundations were there. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From drsalists at gmail.com Tue Feb 13 14:02:56 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 13 Feb 2018 11:02:56 -0800 Subject: Packaging uwsgi flask app for non-programmers? In-Reply-To: References: <605B7863-F542-4D54-A45A-1AD5284807F7@ravnalaska.net> Message-ID: On Tue, Feb 13, 2018 at 9:28 AM, Israel Brewster wrote: > As such, I'm considering three possible solutions: > > 1) Make some sort of installer package that includes the python3 installer > 2) Somehow automate the download and install of Python3, or > 3) re-write my code to be python 2 compatible (since python 2 is included with the OS) > > If anyone has any suggestions on how I could accomplish 1 or 2, I'd appreciate it. Thanks! Would using homebrew help? http://docs.python-guide.org/en/latest/starting/install3/osx/ BTW, you might use curl | bash to get the ball rolling. I wouldn't recommend moving from 3.x to 2.x, unless perhaps you use a common subset. From androidmaroso at gmail.com Tue Feb 13 15:05:58 2018 From: androidmaroso at gmail.com (Maroso Marco) Date: Tue, 13 Feb 2018 12:05:58 -0800 (PST) Subject: Suggestions on programming in Python an email simple client Message-ID: <050fb5ca-4ca7-4ffc-a8ff-6a68f645aae1@googlegroups.com> Hi, what i'm trying to do is develop my own email client, but a simple one. I just want it to connect to a specific email account and read the subject line of messages coming from a certain email address. I then want it to be able to execute the command i wrote on the subject. It would be nice if i could specify two parameters on the subject line first : password; second : command; The first parameter is the password and the second is the command to be executed in the local pc where this program is running. The program will have to show only email coming from a specific account and if the mail is unread then it should read the subject line and find parameters. The first parameter must match my hardcoded password, and the second parameter must be executed on the local pc (example ... c:\windows\notepad.exe) It must check for new emails at a given timing (example ...every 30 seconds ) (the timing can be hardcoded or specified in a separate file) This is it. Any suggestions? I have started doing this (but i never programmed in python before): import imaplib import os import email import email.header plusmail = "emailaddresstobechecked at gmail.com" googlepass = "mypassword" owner = "validsender at gmail.com" M = imaplib.IMAP4_SSL('imap.gmail.com') M.login(plusmail, googlepass) M.select() status, response = M.search(None, '(UNSEEN)') unread_msg_nums = response[0].split() # Print the count of all unread messages print (len(unread_msg_nums)) # Print all unread messages from a certain sender of interest status, response = M.search(None, '(UNSEEN)', '(FROM "%s")' % (owner)) unread_msg_nums = response[0].split() da = [] for e_id in unread_msg_nums: _, response = imap.fetch(e_id, '(BODY.PEEK[HEADER.FIELDS (From Subject)] RFC822.SIZE)') da.append(response[0][1]) # print (da) # Mark them as seen for e_id in unread_msg_nums: imap.store(e_id, '+FLAGS', '\Seen') typ, data = M.search(None, 'From',(owner)) for num in data[0].split(): typ, data = M.fetch(num, '(RFC822)') print ('Message %s\n%s\n' % (num, data[0][1])) M.close() M.logout() I know there are duplicates but i need to figure out how to build this correctly Thanks in advance for any help given. From rosuav at gmail.com Tue Feb 13 15:18:05 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 14 Feb 2018 07:18:05 +1100 Subject: Suggestions on programming in Python an email simple client In-Reply-To: <050fb5ca-4ca7-4ffc-a8ff-6a68f645aae1@googlegroups.com> References: <050fb5ca-4ca7-4ffc-a8ff-6a68f645aae1@googlegroups.com> Message-ID: On Wed, Feb 14, 2018 at 7:05 AM, Maroso Marco wrote: > Hi, > > what i'm trying to do is develop my own email client, but a simple one. > > I just want it to connect to a specific email account and read the subject line of messages coming from a certain email address. > > I then want it to be able to execute the command i wrote on the subject. Be aware that "from" addresses can easily be forged. I suggest whitelisting a set of valid commands and executing only those. If you want a REAL way to manage your home system remotely, I would recommend SSH instead. > It would be nice if i could specify two parameters on the subject line > > first : password; > second : command; > > The first parameter is the password and the second is the command to be executed in the local pc where this program is running. That's an improvement, but since the password is being transmitted in clear text, it won't prevent attacks. It does mean you won't try to execute the subject lines of random spam, though, so accidents will be reduced (or eliminated). > The program will have to show only email coming from a specific account > and if the mail is unread then it should read the subject line and find parameters. > The first parameter must match my hardcoded password, and the second parameter must be executed on the local pc (example ... c:\windows\notepad.exe) > > It must check for new emails at a given timing (example ...every 30 seconds ) > (the timing can be hardcoded or specified in a separate file) > > This is it. > > Any suggestions? Seems perfectly possible. > I have started doing this (but i never programmed in python before): > > > import imaplib > import os > import email > import email.header > > plusmail = "emailaddresstobechecked at gmail.com" > googlepass = "mypassword" > owner = "validsender at gmail.com" > > M = imaplib.IMAP4_SSL('imap.gmail.com') > M.login(plusmail, googlepass) > M.select() > > status, response = M.search(None, '(UNSEEN)') > unread_msg_nums = response[0].split() > > # Print the count of all unread messages > print (len(unread_msg_nums)) > > # Print all unread messages from a certain sender of interest > status, response = M.search(None, '(UNSEEN)', '(FROM "%s")' % (owner)) > unread_msg_nums = response[0].split() > da = [] > for e_id in unread_msg_nums: > _, response = imap.fetch(e_id, '(BODY.PEEK[HEADER.FIELDS (From Subject)] RFC822.SIZE)') > da.append(response[0][1]) > # print (da) > > # Mark them as seen > for e_id in unread_msg_nums: > imap.store(e_id, '+FLAGS', '\Seen') > > > > typ, data = M.search(None, 'From',(owner)) > for num in data[0].split(): > typ, data = M.fetch(num, '(RFC822)') > print ('Message %s\n%s\n' % (num, data[0][1])) > M.close() > M.logout() > > I know there are duplicates but i need to figure out how to build this correctly Seems like a reasonable start. So what's the code doing wrong, or failing to do? Is it failing with an exception? Is it running successfully, but not doing something you want it to do? ChrisA From greg.ewing at canterbury.ac.nz Tue Feb 13 17:08:08 2018 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 14 Feb 2018 11:08:08 +1300 Subject: Respam levels. In-Reply-To: References: Message-ID: Mark Lawrence wrote: > IIRC the same source for the "nospam" stuff of some months ago which I > believe was purely accidental. I have told the site admin about this, hopefully he will be able to put a stop to it again. -- Greg From androidmaroso at gmail.com Tue Feb 13 17:54:23 2018 From: androidmaroso at gmail.com (Maroso Marco) Date: Tue, 13 Feb 2018 14:54:23 -0800 (PST) Subject: Suggestions on programming in Python an email simple client In-Reply-To: References: <050fb5ca-4ca7-4ffc-a8ff-6a68f645aae1@googlegroups.com> Message-ID: <7acb478a-589d-4578-a862-3c827cbcf34b@googlegroups.com> Il giorno marted? 13 febbraio 2018 21:18:37 UTC+1, Chris Angelico ha scritto: > On Wed, Feb 14, 2018 at 7:05 AM, Maroso Marco wrote: > > Hi, > > > > what i'm trying to do is develop my own email client, but a simple one. > > > > I just want it to connect to a specific email account and read the subject line of messages coming from a certain email address. > > > > I then want it to be able to execute the command i wrote on the subject. > > Be aware that "from" addresses can easily be forged. I suggest > whitelisting a set of valid commands and executing only those. If you > want a REAL way to manage your home system remotely, I would recommend > SSH instead. > > > It would be nice if i could specify two parameters on the subject line > > > > first : password; > > second : command; > > > > The first parameter is the password and the second is the command to be executed in the local pc where this program is running. > > That's an improvement, but since the password is being transmitted in > clear text, it won't prevent attacks. It does mean you won't try to > execute the subject lines of random spam, though, so accidents will be > reduced (or eliminated). > > > The program will have to show only email coming from a specific account > > and if the mail is unread then it should read the subject line and find parameters. > > The first parameter must match my hardcoded password, and the second parameter must be executed on the local pc (example ... c:\windows\notepad.exe) > > > > It must check for new emails at a given timing (example ...every 30 seconds ) > > (the timing can be hardcoded or specified in a separate file) > > > > This is it. > > > > Any suggestions? > > Seems perfectly possible. > > > I have started doing this (but i never programmed in python before): > > > > > > import imaplib > > import os > > import email > > import email.header > > > > plusmail = "emailaddresstobechecked at gmail.com" > > googlepass = "mypassword" > > owner = "validsender at gmail.com" > > > > M = imaplib.IMAP4_SSL('imap.gmail.com') > > M.login(plusmail, googlepass) > > M.select() > > > > status, response = M.search(None, '(UNSEEN)') > > unread_msg_nums = response[0].split() > > > > # Print the count of all unread messages > > print (len(unread_msg_nums)) > > > > # Print all unread messages from a certain sender of interest > > status, response = M.search(None, '(UNSEEN)', '(FROM "%s")' % (owner)) > > unread_msg_nums = response[0].split() > > da = [] > > for e_id in unread_msg_nums: > > _, response = imap.fetch(e_id, '(BODY.PEEK[HEADER.FIELDS (From Subject)] RFC822.SIZE)') > > da.append(response[0][1]) > > # print (da) > > > > # Mark them as seen > > for e_id in unread_msg_nums: > > imap.store(e_id, '+FLAGS', '\Seen') > > > > > > > > typ, data = M.search(None, 'From',(owner)) > > for num in data[0].split(): > > typ, data = M.fetch(num, '(RFC822)') > > print ('Message %s\n%s\n' % (num, data[0][1])) > > M.close() > > M.logout() > > > > I know there are duplicates but i need to figure out how to build this correctly > > Seems like a reasonable start. So what's the code doing wrong, or > failing to do? Is it failing with an exception? Is it running > successfully, but not doing something you want it to do? > > ChrisA Hi Chris and thanks for replying. I came up with the following code (thanks for the security suggestions-i'm just testing) : My code actually works fine allthough i have a couple of problems : 1 - once parced the messages and executed the command in the subject line, it keeps looping even if i told to only execute on unread messages 2 - it loops continuosly instead of waiting the correct wait time 3 - i had to add a delete statement so that the program stops looping at least on processing the commands. Can you help me sort out the issue? Maybe it's an indent problem! Here is the code : import imaplib import os import email import email.header import time def mailcontroller(): # Set user, pass and allowed mail for giving commands plusmail = "emailaddresstobechecked at gmail.com" googlepass = "thesecretpassword" captain = "theallowedemailaddress at gmail.com" # Set vars for IMAP access M = imaplib.IMAP4_SSL('imap.gmail.com') M.login(plusmail, googlepass) M.select() # Set search on UNSEEN messages status, response = M.search(None, '(UNSEEN)') unread_msg_nums = response[0].split() # Mark as read for e_id in unread_msg_nums: M.store(e_id, '+FLAGS', '\Seen') # cycle messages sent from autorized email address typ, data = M.search(None, 'From',(captain)) for num in data[0].split(): typ, data = M.fetch(num, '(RFC822)') msg = email.message_from_string(data[0][1]) decode = email.header.decode_header(msg['Subject'])[0] subject = unicode(decode[0]) comando = subject if googlepass in subject: print 'Message %s: %s' % (num, subject) # Split subject line googlepass,comando = subject.split(";") # Print command print comando # Execute command os.system(comando) M.store(num, '+FLAGS', '\\Deleted') M.expunge() elif googlepass not in subject: print "Nope" M.close() M.logout() time.sleep(10) while True: mailcontroller() THanks in advance :-) From israel at ravnalaska.net Tue Feb 13 18:10:58 2018 From: israel at ravnalaska.net (Israel Brewster) Date: Tue, 13 Feb 2018 14:10:58 -0900 Subject: Packaging uwsgi flask app for non-programmers? In-Reply-To: References: <605B7863-F542-4D54-A45A-1AD5284807F7@ravnalaska.net> Message-ID: <3BC89213-92D8-4F49-B173-AC504BCD8E7C@ravnalaska.net> > On Feb 13, 2018, at 10:02 AM, Dan Stromberg wrote: > > On Tue, Feb 13, 2018 at 9:28 AM, Israel Brewster wrote: >> As such, I'm considering three possible solutions: >> >> 1) Make some sort of installer package that includes the python3 installer >> 2) Somehow automate the download and install of Python3, or >> 3) re-write my code to be python 2 compatible (since python 2 is included with the OS) >> >> If anyone has any suggestions on how I could accomplish 1 or 2, I'd appreciate it. Thanks! > > Would using homebrew help? > > http://docs.python-guide.org/en/latest/starting/install3/osx/ That's a thought. I could offer the user the option of either a) automatically installing homebrew and then installing python3 via homebrew, or b) manually downloading and running the python3 installer. > > BTW, you might use curl | bash to get the ball rolling. On that note, is there a fixed url that will always get the latest python3 installer? Of course, I might not want to do that, for example after 3.7 or 4.0 (I know, not for a while) is released, just in case something breaks with a newer release. > > I wouldn't recommend moving from 3.x to 2.x, unless perhaps you use a > common subset. Yeah, that idea kinda left a sour taste in my mouth, but I figured I'd throw it out there as it would solve the python install issue. ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From python at mrabarnett.plus.com Tue Feb 13 18:35:15 2018 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 13 Feb 2018 23:35:15 +0000 Subject: Suggestions on programming in Python an email simple client In-Reply-To: <7acb478a-589d-4578-a862-3c827cbcf34b@googlegroups.com> References: <050fb5ca-4ca7-4ffc-a8ff-6a68f645aae1@googlegroups.com> <7acb478a-589d-4578-a862-3c827cbcf34b@googlegroups.com> Message-ID: <38a01a6d-6aaa-52d0-eac3-a58f1b6c8c97@mrabarnett.plus.com> On 2018-02-13 22:54, Maroso Marco wrote: > Il giorno marted? 13 febbraio 2018 21:18:37 UTC+1, Chris Angelico ha scritto: >> On Wed, Feb 14, 2018 at 7:05 AM, Maroso Marco wrote: >> > Hi, >> > >> > what i'm trying to do is develop my own email client, but a simple one. >> > >> > I just want it to connect to a specific email account and read the subject line of messages coming from a certain email address. >> > >> > I then want it to be able to execute the command i wrote on the subject. >> >> Be aware that "from" addresses can easily be forged. I suggest >> whitelisting a set of valid commands and executing only those. If you >> want a REAL way to manage your home system remotely, I would recommend >> SSH instead. >> >> > It would be nice if i could specify two parameters on the subject line >> > >> > first : password; >> > second : command; >> > >> > The first parameter is the password and the second is the command to be executed in the local pc where this program is running. >> >> That's an improvement, but since the password is being transmitted in >> clear text, it won't prevent attacks. It does mean you won't try to >> execute the subject lines of random spam, though, so accidents will be >> reduced (or eliminated). >> >> > The program will have to show only email coming from a specific account >> > and if the mail is unread then it should read the subject line and find parameters. >> > The first parameter must match my hardcoded password, and the second parameter must be executed on the local pc (example ... c:\windows\notepad.exe) >> > >> > It must check for new emails at a given timing (example ...every 30 seconds ) >> > (the timing can be hardcoded or specified in a separate file) >> > >> > This is it. >> > >> > Any suggestions? >> >> Seems perfectly possible. >> >> > I have started doing this (but i never programmed in python before): >> > >> > >> > import imaplib >> > import os >> > import email >> > import email.header >> > >> > plusmail = "emailaddresstobechecked at gmail.com" >> > googlepass = "mypassword" >> > owner = "validsender at gmail.com" >> > >> > M = imaplib.IMAP4_SSL('imap.gmail.com') >> > M.login(plusmail, googlepass) >> > M.select() >> > >> > status, response = M.search(None, '(UNSEEN)') >> > unread_msg_nums = response[0].split() >> > >> > # Print the count of all unread messages >> > print (len(unread_msg_nums)) >> > >> > # Print all unread messages from a certain sender of interest >> > status, response = M.search(None, '(UNSEEN)', '(FROM "%s")' % (owner)) >> > unread_msg_nums = response[0].split() >> > da = [] >> > for e_id in unread_msg_nums: >> > _, response = imap.fetch(e_id, '(BODY.PEEK[HEADER.FIELDS (From Subject)] RFC822.SIZE)') >> > da.append(response[0][1]) >> > # print (da) >> > >> > # Mark them as seen >> > for e_id in unread_msg_nums: >> > imap.store(e_id, '+FLAGS', '\Seen') >> > >> > >> > >> > typ, data = M.search(None, 'From',(owner)) >> > for num in data[0].split(): >> > typ, data = M.fetch(num, '(RFC822)') >> > print ('Message %s\n%s\n' % (num, data[0][1])) >> > M.close() >> > M.logout() >> > >> > I know there are duplicates but i need to figure out how to build this correctly >> >> Seems like a reasonable start. So what's the code doing wrong, or >> failing to do? Is it failing with an exception? Is it running >> successfully, but not doing something you want it to do? >> >> ChrisA > > Hi Chris and thanks for replying. > > I came up with the following code (thanks for the security suggestions-i'm just testing) : > > My code actually works fine allthough i have a couple of problems : > > 1 - once parced the messages and executed the command in the subject line, it keeps looping even if i told to only execute on unread messages > 2 - it loops continuosly instead of waiting the correct wait time > 3 - i had to add a delete statement so that the program stops looping at least on processing the commands. > > Can you help me sort out the issue? Maybe it's an indent problem! > > > Here is the code : > > import imaplib > import os > import email > import email.header > import time > > def mailcontroller(): > > # Set user, pass and allowed mail for giving commands > plusmail = "emailaddresstobechecked at gmail.com" > googlepass = "thesecretpassword" > captain = "theallowedemailaddress at gmail.com" > > # Set vars for IMAP access > M = imaplib.IMAP4_SSL('imap.gmail.com') > M.login(plusmail, googlepass) > M.select() > > # Set search on UNSEEN messages > status, response = M.search(None, '(UNSEEN)') > unread_msg_nums = response[0].split() > > > # Mark as read > for e_id in unread_msg_nums: > M.store(e_id, '+FLAGS', '\Seen') > Here you iterate through all of the messages: > # cycle messages sent from autorized email address > typ, data = M.search(None, 'From',(captain)) > for num in data[0].split(): > typ, data = M.fetch(num, '(RFC822)') > msg = email.message_from_string(data[0][1]) > decode = email.header.decode_header(msg['Subject'])[0] > subject = unicode(decode[0]) > comando = subject > These lines are indented the same as the 'for' loop above, so they'll executed only the last message. If there are no messages, 'subject' won't be defined. > if googlepass in subject: > print 'Message %s: %s' % (num, subject) > > # Split subject line > googlepass,comando = subject.split(";") > # Print command > print comando > # Execute command > os.system(comando) > > M.store(num, '+FLAGS', '\\Deleted') There's no need to use 'expunge' here because those messages that are marked for deletion will be expunged anyway when the mailbox is closed: > M.expunge() > Why not just "else:" here? > elif googlepass not in subject: > print "Nope" > > M.close() > M.logout() > Actually, the sequence should be more like: login for each message: execute if necessary close and logout > > time.sleep(10) > > while True: > mailcontroller() > It should really be sleeping between calls. > THanks in advance :-) > From tkadm30 at yandex.com Tue Feb 13 18:41:05 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 13 Feb 2018 18:41:05 -0500 Subject: Django-hotsauce 1.0 commercial edition looking for beta testers! Message-ID: <21340642-c3ef-c50e-e49b-6c86c8e8849a@yandex.com> Hello everyone, Django-hotsauce 1.0 commercial edition (LTS) is now available for preorder :) Checkout: https://www.livestore.ca/product/django-hotsauce/ I'm also looking for expert Django and Python programmers to test and review the design and API of Django-hotsauce 1.0 commercial edition in order to develop a low-cost, high-performance, Django alternative microframework to develop scalable web apps in Python and WSGI. Furthermore, i will always be happy to support a free version of Django-hotsauce in order to drive the development and research of Python/Django web programming. :) Final note, the Paypal payment gateway is experimental. I might one day decide to switch to monero or flashcoin... I'm not really sure it's a good idea yet. What do you think? Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From steve+comp.lang.python at pearwood.info Tue Feb 13 19:13:06 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 14 Feb 2018 00:13:06 +0000 (UTC) Subject: Defer, ensure library is loaded References: <2a0adcd7-4331-4ae1-bc6e-d0a5df86b69f@googlegroups.com> Message-ID: On Wed, 14 Feb 2018 03:08:07 +1100, Chris Angelico wrote: [...] > import psycopg2 > > If it's already been imported, Python will go fetch it straight from the > cache, so it's fast. Since Jason is talking about "a variety of scripts", it is quite possible that the sys.modules cache will not save him. Consider a scenario of a dozen scripts, each importing psycopg2. Since each script runs in its own interpreter process, sys.modules is cleared from one run to the next. For scripts run by hand, this rarely matters. (After all, the time to run the script is often less than the time it takes to type in the command to run and hit Enter.) But for scripts called by other scripts, this can sometimes cause a significant slowdown. (My understanding is that this is one of the reasons for the Apache module "mod_python", to ensure a single Apache process runs a single, long-running Python interpreter for any Python scripts, rather than starting and stopping a new interpreter like ordinary CGI scripts would do.) I must admit though, I don't really understand Jason's problem or whether your answer is even relevant. -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 13 19:25:25 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 14 Feb 2018 00:25:25 +0000 (UTC) Subject: Defer, ensure library is loaded References: <2a0adcd7-4331-4ae1-bc6e-d0a5df86b69f@googlegroups.com> Message-ID: On Tue, 13 Feb 2018 08:02:38 -0800, Jason wrote: > I have a variety of scripts that import some large libraries, and rather > than create a million little scripts with specific imports, I'd like to > so something like > > psycopg2 = ensure_imported (psycopg2) > > This way, regardless of invocation I can know psycopg2 is loaded, if it > hasn't already been loaded. If I just do a normal import 95% of the time > I'll be waiting time with a meaningless import. > > > Can I do that somehow? Jason, I'm afraid I don't understand your question. I don't understand what "ensure_imported" is supposed to do, or how it is different from a regular import, or where you are supposed to put it. If you have a "million scripts" that look like this: # count down from script 1000000 to 1... import psycopg2 ... # script 999999 import psycopg2 ... # script 999998 import psycopg2 ... etc, how is that different from a million scripts that look like this? # count down from script 1000000 to 1... psycopg2 = ensure_imported(psycopg2) ... Aside from the ensure_imported version being about double as much typing. Nor do I understand what you mean by having to spend 95% of your time waiting for a meaningless import. If the import is "meaningless", that could only possibly mean that you aren't using the imported module. So the solution to that is to simply *not* import it. You shouldn't start off every module by importing a bazillion modules that you don't use. Import only what you actually use. If you explain your problem a little better, we can possibly help you solve it. Oh, one last thing... you cannot write something like this: foomodule = ensure_imported(foomodule) because the interpreter has to evaluate the value of the name "foomodule" on the right hand side of the assignment, and since it hasn't been imported yet, you will get a NameError. You have to write: foomodule = ensure_imported("foomodule") # note the quotation marks to have it work. -- Steve From ned at nedbatchelder.com Tue Feb 13 19:54:33 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 13 Feb 2018 19:54:33 -0500 Subject: Django-hotsauce 1.0 commercial edition looking for beta testers! In-Reply-To: <21340642-c3ef-c50e-e49b-6c86c8e8849a@yandex.com> References: <21340642-c3ef-c50e-e49b-6c86c8e8849a@yandex.com> Message-ID: On 2/13/18 6:41 PM, Etienne Robillard wrote: > Hello everyone, > > Django-hotsauce 1.0 commercial edition (LTS) is now available for > preorder :) > > Checkout: https://www.livestore.ca/product/django-hotsauce/ > > I'm also looking for expert Django and Python programmers to test and > review the design and API of Django-hotsauce 1.0 commercial edition in > order to develop a low-cost, high-performance, Django alternative > microframework to develop scalable web apps in Python and WSGI. > Furthermore, i will always be happy to support a free version of > Django-hotsauce in order to drive the development and research of > Python/Django web programming. :) > > Final note, the Paypal payment gateway is experimental. I might one > day decide to switch to monero or flashcoin... I'm not really sure > it's a good idea yet. There are a number of things that are confusing about this.? You should state clearly what django-hotsauce is. You say "Django alternative microframework."? Usually people use "Django" and "microframework" as opposites, not something you can combine together. You say there will be a free version.? Does it already exist? Link to it. Finally, you seem to be asking experts to volunteer their time to help you design and test software that you will then sell.? This seems unrealistic at best, and exploitive or unfair at worst. --Ned. From tkadm30 at yandex.com Wed Feb 14 04:06:20 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 14 Feb 2018 04:06:20 -0500 Subject: Django-hotsauce 1.0 commercial edition looking for beta testers! In-Reply-To: References: <21340642-c3ef-c50e-e49b-6c86c8e8849a@yandex.com> Message-ID: <78621c79-16ef-21fe-23af-e6f4147d012d@yandex.com> Hi Ned Le 2018-02-13 ? 19:54, Ned Batchelder a ?crit?: > > There are a number of things that are confusing about this.? You > should state clearly what django-hotsauce is. You say "Django > alternative microframework."? Usually people use "Django" and > "microframework" as opposites, not something you can combine together. Sorry for being unclear. I meant "Django-based alternative microframework". :-) I plan to commercialize a Django extension for allowing pragmatic and high-performance WSGI programming *on top* of Django. > > You say there will be a free version.? Does it already exist? Link to it. https://www.isotopesoftware.ca/software/django-hotsauce/ https://pypi.python.org/pypi/Django-hotsauce/0.9.0 > > Finally, you seem to be asking experts to volunteer their time to help > you design and test software that you will then sell.? This seems > unrealistic at best, and exploitive or unfair at worst. > Yes. I'm really looking for serious people and/or companies to try/test/review django-hotsauce 1.0 commercial edition for *commercial purposes* in exchange of a small fee. > --Ned. Best regards, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From as at sci.fi Wed Feb 14 04:59:31 2018 From: as at sci.fi (Anssi Saari) Date: Wed, 14 Feb 2018 11:59:31 +0200 Subject: Respam levels. References: Message-ID: Mark Lawrence writes: >> This may be the result of a misconfigured spam filter, or an actual spam >> attack; anyway, I've now filtered news.bbs.geek.nz from my feed. >> > IIRC the same source for the "nospam" stuff of some months ago which I > believe was purely accidental. It's kinda funny, back in the 90s Usenet was spammed every now and again by various misconfigured BBS's running a Usenet gateway. Usually sending all messages back from every group to every group. I thought BBS's are fairly dead and Usenet is mostly gone too but apparently this downside has made a comeback. From none at gmail.com Wed Feb 14 07:46:24 2018 From: none at gmail.com (ast) Date: Wed, 14 Feb 2018 13:46:24 +0100 Subject: Old format with % Message-ID: <5a842fa4$0$20431$426a74cc@news.free.fr> Hello It seems that caracter % can't be escaped >>>"test %d %" % 7 ValueError: incomplete format >>>"test %d \%" % 7 ValueError: incomplete format >>>"test %d" % 7 + "%" 'test 7%' # OK But is there a way to escape a % ? thx From none at gmail.com Wed Feb 14 07:54:56 2018 From: none at gmail.com (ast) Date: Wed, 14 Feb 2018 13:54:56 +0100 Subject: Old format with % In-Reply-To: <5a842fa4$0$20431$426a74cc@news.free.fr> References: <5a842fa4$0$20431$426a74cc@news.free.fr> Message-ID: <5a8431a4$0$3868$426a34cc@news.free.fr> Le 14/02/2018 ? 13:46, ast a ?crit?: > Hello > > It seems that caracter % can't be escaped > > >>>"test %d %" % 7 > ValueError: incomplete format > > >>>"test %d \%" % 7 > ValueError: incomplete format > > >>>"test %d" % 7 + "%" > 'test 7%'? # OK > > But is there a way to escape a % ? > > thx Found, double % to escape it >>>"test %d%%" % 7 'test 7%' From tjreedy at udel.edu Wed Feb 14 14:52:07 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 14 Feb 2018 14:52:07 -0500 Subject: Old format with % In-Reply-To: <5a8431a4$0$3868$426a34cc@news.free.fr> References: <5a842fa4$0$20431$426a74cc@news.free.fr> <5a8431a4$0$3868$426a34cc@news.free.fr> Message-ID: On 2/14/2018 7:54 AM, ast wrote: > Le 14/02/2018 ? 13:46, ast a ?crit?: >> Hello >> >> It seems that caracter % can't be escaped >> >> ?>>>"test %d %" % 7 >> ValueError: incomplete format >> >> ?>>>"test %d \%" % 7 >> ValueError: incomplete format >> >> ?>>>"test %d" % 7 + "%" >> 'test 7%'? # OK >> >> But is there a way to escape a % ? >> >> thx > > Found, double % to escape it > > >>>"test %d%%" % 7 > 'test 7%' Same with { and } in new format and f strings. >>> a = 3 >>> f'{{x:{a}}}' '{x:3}' -- Terry Jan Reedy From darcy at VybeNetworks.com Wed Feb 14 16:24:23 2018 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Wed, 14 Feb 2018 15:24:23 -0600 Subject: What F strings should have been Message-ID: <27525c94-2167-ea96-4167-06b830923e2e@VybeNetworks.com> A recent post by Terry Jan Reedy got me thinking about formatting. I like the new(ish) format method for strings and I see some value in F strings but it only works well with locals. Anything more starts getting messier than format() and it is supposed to be cleaner. Also, I find that I tend to re-use format strings and wish there was a formatting string option. Here is what I came up with. class FSTR(str): def __call__(self, *args): return self.format(*args) And here is how it could be used. s = FSTR("ABC {1} {0} {2[x]}") ... print(s(1, 2, dict(x=3))) Too bad that it is too late to assign f'' to that function. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From enerel.amgalan at yahoo.com Wed Feb 14 21:48:00 2018 From: enerel.amgalan at yahoo.com (Enerel Amgalan) Date: Thu, 15 Feb 2018 02:48:00 +0000 (UTC) Subject: "Programs" folder not found. References: <1852393400.1245525.1518662880464.ref@mail.yahoo.com> Message-ID: <1852393400.1245525.1518662880464@mail.yahoo.com> Hello! So I downloaded ?Python? program in C:>Users>(my name)>AppData>Local>Programs>Python.And then in ?Local? folder I can?t find ?Programs? folder,but it says it downloaded in ?Programs?.So can you help me. From kamikaze.is.waiting.you at gmail.com Wed Feb 14 22:40:08 2018 From: kamikaze.is.waiting.you at gmail.com (Oleg Korsak) Date: Thu, 15 Feb 2018 05:40:08 +0200 Subject: Python GIL vs CAS Message-ID: Hi. While hearing about GIL every time... is there any real reason why CAS doesn't help to solve this problem? https://en.wikipedia.org/wiki/Compare-and-swap From rajindermohan001 at gmail.com Thu Feb 15 02:07:00 2018 From: rajindermohan001 at gmail.com (rajindermohan001 at gmail.com) Date: Wed, 14 Feb 2018 23:07:00 -0800 (PST) Subject: Why there is no "setdefaultencoding" in sys module? In-Reply-To: <576f217a-d4f2-4f56-b631-a393e31e0151@q21g2000prm.googlegroups.com> References: <576f217a-d4f2-4f56-b631-a393e31e0151@q21g2000prm.googlegroups.com> Message-ID: <0142e71a-e759-4870-8c2c-d11f9407d20c@googlegroups.com> On Friday, July 9, 2010 at 9:28:35 PM UTC+5:30, crow wrote: > Hi, everyone > > I'm a new hand at python. > > I tried to set system default encoding by using > > "import sys; sys.setdefaultencoding('utf-f')", > > but I got error message: > > >>> sys.setdefaultencoding('utf-8') > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'module' object has no attribute 'setdefaultencoding' > > Then I checked dir(sys), seems there was no function named > "setdefaultencoding" in "sys" module. But in python's document, it > said I should use sys.setdefaultencoding. > > So, my questions: why there is no setdefaultencoding in sys module? if > I want to change system's default encoding, what should I do? > > Thanks in advance Becuase you are using python 3.setdefaultencoding is no longer supportedin python 3. python3automatically have utf-8 in its configuration by default. so you don't have to use it. just remove the line. From __peter__ at web.de Thu Feb 15 03:56:07 2018 From: __peter__ at web.de (Peter Otten) Date: Thu, 15 Feb 2018 09:56:07 +0100 Subject: What F strings should have been References: <27525c94-2167-ea96-4167-06b830923e2e@VybeNetworks.com> Message-ID: D'Arcy Cain wrote: > A recent post by Terry Jan Reedy got me thinking about formatting. I > like the new(ish) format method for strings and I see some value in F > strings but it only works well with locals. Anything more starts > getting messier than format() and it is supposed to be cleaner. Also, I > find that I tend to re-use format strings and wish there was a > formatting string option. Here is what I came up with. > > class FSTR(str): > def __call__(self, *args): > return self.format(*args) > > And here is how it could be used. > > s = FSTR("ABC {1} {0} {2[x]}") This can be simplified to s = "ABC {1} {0} {2[x]}".format > ... > print(s(1, 2, dict(x=3))) > > Too bad that it is too late to assign f'' to that function. I must be missing something. How would you simplify or improve the readability of the above with your version of f"..."? From gbs.deadeye at gmail.com Thu Feb 15 04:14:17 2018 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Thu, 15 Feb 2018 09:14:17 +0000 Subject: Old format with % In-Reply-To: References: <5a842fa4$0$20431$426a74cc@news.free.fr> <5a8431a4$0$3868$426a34cc@news.free.fr> Message-ID: It can be escaped: "test %d %%" % 7 Terry Reedy schrieb am Mi., 14. Feb. 2018 um 20:53 Uhr: > On 2/14/2018 7:54 AM, ast wrote: > > Le 14/02/2018 ? 13:46, ast a ?crit : > >> Hello > >> > >> It seems that caracter % can't be escaped > >> > >> >>>"test %d %" % 7 > >> ValueError: incomplete format > >> > >> >>>"test %d \%" % 7 > >> ValueError: incomplete format > >> > >> >>>"test %d" % 7 + "%" > >> 'test 7%' # OK > >> > >> But is there a way to escape a % ? > >> > >> thx > > > > Found, double % to escape it > > > > >>>"test %d%%" % 7 > > 'test 7%' > > Same with { and } in new format and f strings. > >>> a = 3 > >>> f'{{x:{a}}}' > '{x:3}' > > -- > Terry Jan Reedy > > > -- > https://mail.python.org/mailman/listinfo/python-list > From gbs.deadeye at gmail.com Thu Feb 15 04:23:08 2018 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Thu, 15 Feb 2018 09:23:08 +0000 Subject: Regex on a Dictionary In-Reply-To: References: <4557e975-7964-446e-b5f6-d8954fe3dd93@googlegroups.com> <423885d6-54b7-4e10-8e8a-dde2681154e0@googlegroups.com> Message-ID: Hello, this question also came up there: https://python-forum.io/Thread-Working-with-Dict-Object Greetings Andre From darcy at VybeNetworks.com Thu Feb 15 05:51:43 2018 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Thu, 15 Feb 2018 04:51:43 -0600 Subject: What F strings should have been In-Reply-To: References: <27525c94-2167-ea96-4167-06b830923e2e@VybeNetworks.com> Message-ID: <2f028b00-c8bf-7fb2-5ba8-ab9494f3742b@VybeNetworks.com> On 02/15/18 02:56, Peter Otten wrote: >> class FSTR(str): >> def __call__(self, *args): >> return self.format(*args) >> >> And here is how it could be used. >> >> s = FSTR("ABC {1} {0} {2[x]}") > > This can be simplified to > > s = "ABC {1} {0} {2[x]}".format Hmm. Hadn't thought of that. >> print(s(1, 2, dict(x=3))) >> >> Too bad that it is too late to assign f'' to that function. > > I must be missing something. How would you simplify or improve the > readability of the above with your version of f"..."? s = f"ABC {1} {0} {2[x]}" -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From sjsumitj at gmail.com Thu Feb 15 06:07:01 2018 From: sjsumitj at gmail.com (Sum J) Date: Thu, 15 Feb 2018 03:07:01 -0800 (PST) Subject: Getting "ValueError: need more than 1 value to unpack" while trying to read a value from dictionary in python Message-ID: <4eef21f4-ef17-4c59-8c89-59a6a96c43cb@googlegroups.com> Below is my code. Here I want to read the "ip address" from s s= ''' Power On Enabled = On State: connected Radio Module: Unknown noise: -097 signalStrength: -046 ip address: 192.168.75.147 subnet mask: 255.255.255.0 IPv4 address configured by DHCP Mac Addr: ac:e2:d3:32:00:5a Mode: infrastrastructure ssid: Cloudlab Channel: 1 Regulatory: World Safe Authencation: WPA2/PSK Encryption: AES or TKIP ''' s = s.replace("=",":") # s = s.strip() print s d = {} for i in s: key, val = i.split(":") d[key] = val.strip() print d print d["ip address"] Getting below error : key, val = i.split(":") ValueError: need more than 1 value to unpack From acharbly at gmail.com Thu Feb 15 06:17:08 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Thu, 15 Feb 2018 16:47:08 +0530 Subject: Getting "ValueError: need more than 1 value to unpack" while trying to read a value from dictionary in python In-Reply-To: References: <4eef21f4-ef17-4c59-8c89-59a6a96c43cb@googlegroups.com> Message-ID: IPv4 address configured by DHCP It doesn't find key and value for above string. Solution.. Use one complete string per line On 15 Feb 2018 4:41 pm, "Sum J" wrote: Below is my code. Here I want to read the "ip address" from s s= ''' Power On Enabled = On State: connected Radio Module: Unknown noise: -097 signalStrength: -046 ip address: 192.168.75.147 subnet mask: 255.255.255.0 IPv4 address configured by DHCP Mac Addr: ac:e2:d3:32:00:5a Mode: infrastrastructure ssid: Cloudlab Channel: 1 Regulatory: World Safe Authencation: WPA2/PSK Encryption: AES or TKIP ''' s = s.replace("=",":") # s = s.strip() print s d = {} for i in s: key, val = i.split(":") d[key] = val.strip() print d print d["ip address"] Getting below error : key, val = i.split(":") ValueError: need more than 1 value to unpack -- https://mail.python.org/mailman/listinfo/python-list From kwpolska at gmail.com Thu Feb 15 06:25:14 2018 From: kwpolska at gmail.com (Chris Warrick) Date: Thu, 15 Feb 2018 12:25:14 +0100 Subject: Getting "ValueError: need more than 1 value to unpack" while trying to read a value from dictionary in python In-Reply-To: <4eef21f4-ef17-4c59-8c89-59a6a96c43cb@googlegroups.com> References: <4eef21f4-ef17-4c59-8c89-59a6a96c43cb@googlegroups.com> Message-ID: On 15 February 2018 at 12:07, Sum J wrote: > Below is my code. Here I want to read the "ip address" from s > > > s= ''' > Power On Enabled = On > State: connected > Radio Module: Unknown > noise: -097 > signalStrength: -046 > ip address: 192.168.75.147 > subnet mask: 255.255.255.0 > IPv4 address configured by DHCP > Mac Addr: ac:e2:d3:32:00:5a > Mode: infrastrastructure > ssid: Cloudlab > Channel: 1 > Regulatory: World Safe > Authencation: WPA2/PSK > Encryption: AES or TKIP > ''' > > s = s.replace("=",":") > # s = s.strip() > print s > > d = {} > for i in s: > key, val = i.split(":") > d[key] = val.strip() > > print d > print d["ip address"] > > > Getting below error : > key, val = i.split(":") > ValueError: need more than 1 value to unpack > -- > https://mail.python.org/mailman/listinfo/python-list If you iterate over a string, you are iterating over individual characters. Instead, you need to split it into lines, first stripping whitespace (starts and ends with an empty line). s = s.strip().replace("=",":") print s d = {} for i in s.split('\n'): try: key, val = i.split(":") d[key.strip()] = val.strip() except ValueError: print "no key:value pair found in", i (PS. please switch to Python 3) -- Chris Warrick PGP: 5EAAEA16 From rosuav at gmail.com Thu Feb 15 09:35:00 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 16 Feb 2018 01:35:00 +1100 Subject: Python GIL vs CAS In-Reply-To: References: Message-ID: On Thu, Feb 15, 2018 at 2:40 PM, Oleg Korsak wrote: > Hi. While hearing about GIL every time... is there any real reason why CAS > doesn't help to solve this problem? > > https://en.wikipedia.org/wiki/Compare-and-swap Because the GIL is not a problem. It's a feature. Before you ask about solutions, you need to clarify what you are calling a problem :) ChrisA From gbs.deadeye at gmail.com Thu Feb 15 10:48:00 2018 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Thu, 15 Feb 2018 15:48:00 +0000 Subject: "Programs" folder not found. In-Reply-To: <1852393400.1245525.1518662880464@mail.yahoo.com> References: <1852393400.1245525.1518662880464.ref@mail.yahoo.com> <1852393400.1245525.1518662880464@mail.yahoo.com> Message-ID: Look in %localappdata%\Programs\Python Enerel Amgalan via Python-list schrieb am Do., 15. Feb. 2018 um 14:05 Uhr: > > Hello! So I downloaded ?Python? program in C:>Users>(my > name)>AppData>Local>Programs>Python.And then in ?Local? folder I can?t find > ?Programs? folder,but it says it downloaded in ?Programs?.So can you help > me. > -- > https://mail.python.org/mailman/listinfo/python-list > From tkadm30 at yandex.com Thu Feb 15 11:14:30 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Thu, 15 Feb 2018 11:14:30 -0500 Subject: How to benchmark a HTTP connection with requests module? Message-ID: <2c77caf1-63db-da7d-865d-df94d0661d48@yandex.com> Hi, Is it possible to emulate a range of concurrent connections to a http server with the requests module? import os import requests from test_support import unittest class HTTPBenchmarkTestCase(unittest.TestCase): ??? url = 'http://localhost/benchmark/' ??? def setUp(self): ??????? pass ??? def test_benchmark_concurrency(self): # XXX it would be really nice to execute 10 concurrent requests # with 10 separated threads/connections. ??????? r = requests.get(self.url) ??????? self.assertEqual(r.status_code, 200) ??? def tearDown(self): ??????? pass ??? def runTest(self): ??????? pass What do you think? Etienne From ned at nedbatchelder.com Thu Feb 15 11:27:34 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 15 Feb 2018 11:27:34 -0500 Subject: Python GIL vs CAS In-Reply-To: References: Message-ID: On 2/15/18 9:35 AM, Chris Angelico wrote: > On Thu, Feb 15, 2018 at 2:40 PM, Oleg Korsak > wrote: >> Hi. While hearing about GIL every time... is there any real reason why CAS >> doesn't help to solve this problem? >> >> https://en.wikipedia.org/wiki/Compare-and-swap > Because the GIL is not a problem. It's a feature. Before you ask about > solutions, you need to clarify what you are calling a problem :) > Let's not overstate the case.? The GIL is not a feature, it's an effective solution to a problem.? The Python interpreter has data that is shared among threads, including the state of the interpreter, and all of the reference counts of all of the Python objects.? When that data is modified, it must be done in a thread-safe way.? The GIL is a simple and effective solution to doing that correctly. Oleg, CAS is a processor primitive that can be used to implement synchronization tools, including locks like the GIL.? You can't replace the GIL with CAS, since they are not equivalent.? You'll need to flesh out your idea about how CAS can help solve the "mutating shared interpreter state" problem that the GIL solves. --Ned. From ian.g.kelly at gmail.com Thu Feb 15 11:47:13 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 15 Feb 2018 09:47:13 -0700 Subject: What F strings should have been In-Reply-To: <27525c94-2167-ea96-4167-06b830923e2e@VybeNetworks.com> References: <27525c94-2167-ea96-4167-06b830923e2e@VybeNetworks.com> Message-ID: On Wed, Feb 14, 2018 at 2:24 PM, D'Arcy Cain wrote: > A recent post by Terry Jan Reedy got me thinking about formatting. I > like the new(ish) format method for strings and I see some value in F > strings but it only works well with locals. Anything more starts > getting messier than format() and it is supposed to be cleaner. Also, I > find that I tend to re-use format strings and wish there was a > formatting string option. Here is what I came up with. > > class FSTR(str): > def __call__(self, *args): > return self.format(*args) > > And here is how it could be used. > > s = FSTR("ABC {1} {0} {2[x]}") > ... > print(s(1, 2, dict(x=3))) > > Too bad that it is too late to assign f'' to that function. Someday far in the future Python will be remembered as a language that people in the before-time used to format strings. From rosuav at gmail.com Thu Feb 15 12:12:33 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 16 Feb 2018 04:12:33 +1100 Subject: Python GIL vs CAS In-Reply-To: References: Message-ID: On Fri, Feb 16, 2018 at 3:27 AM, Ned Batchelder wrote: > On 2/15/18 9:35 AM, Chris Angelico wrote: >> >> On Thu, Feb 15, 2018 at 2:40 PM, Oleg Korsak >> wrote: >>> >>> Hi. While hearing about GIL every time... is there any real reason why >>> CAS >>> doesn't help to solve this problem? >>> >>> https://en.wikipedia.org/wiki/Compare-and-swap >> >> Because the GIL is not a problem. It's a feature. Before you ask about >> solutions, you need to clarify what you are calling a problem :) >> > > Let's not overstate the case. The GIL is not a feature, it's an effective > solution to a problem. The Python interpreter has data that is shared among > threads, including the state of the interpreter, and all of the reference > counts of all of the Python objects. When that data is modified, it must be > done in a thread-safe way. The GIL is a simple and effective solution to > doing that correctly. Okay, I kinda exaggerated. But the GIL isn't a "problem". As you say, it's a simple way to achieve an important goal. Removing the GIL from CPython *has* been done, more than once. Each time, the alternative gave some performance improvements, and a number of penalties. ChrisA From sjsumitj at gmail.com Thu Feb 15 13:42:01 2018 From: sjsumitj at gmail.com (Sum J) Date: Fri, 16 Feb 2018 00:12:01 +0530 Subject: Getting "ValueError: need more than 1 value to unpack" while trying to read a value from dictionary in python In-Reply-To: References: <4eef21f4-ef17-4c59-8c89-59a6a96c43cb@googlegroups.com> Message-ID: Thanks Chris :) Its working now. Regards, Sumit On Thu, Feb 15, 2018 at 4:55 PM, Chris Warrick wrote: > On 15 February 2018 at 12:07, Sum J wrote: > > Below is my code. Here I want to read the "ip address" from s > > > > > > s= ''' > > Power On Enabled = On > > State: connected > > Radio Module: Unknown > > noise: -097 > > signalStrength: -046 > > ip address: 192.168.75.147 > > subnet mask: 255.255.255.0 > > IPv4 address configured by DHCP > > Mac Addr: ac:e2:d3:32:00:5a > > Mode: infrastrastructure > > ssid: Cloudlab > > Channel: 1 > > Regulatory: World Safe > > Authencation: WPA2/PSK > > Encryption: AES or TKIP > > ''' > > > > s = s.replace("=",":") > > # s = s.strip() > > print s > > > > d = {} > > for i in s: > > key, val = i.split(":") > > d[key] = val.strip() > > > > print d > > print d["ip address"] > > > > > > Getting below error : > > key, val = i.split(":") > > ValueError: need more than 1 value to unpack > > -- > > https://mail.python.org/mailman/listinfo/python-list > > If you iterate over a string, you are iterating over individual > characters. Instead, you need to split it into lines, first stripping > whitespace (starts and ends with an empty line). > > s = s.strip().replace("=",":") > print s > > d = {} > for i in s.split('\n'): > try: > key, val = i.split(":") > d[key.strip()] = val.strip() > except ValueError: > print "no key:value pair found in", i > > > (PS. please switch to Python 3) > > -- > Chris Warrick > PGP: 5EAAEA16 > From python at mrabarnett.plus.com Thu Feb 15 13:59:06 2018 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 15 Feb 2018 18:59:06 +0000 Subject: Getting "ValueError: need more than 1 value to unpack" while trying to read a value from dictionary in python In-Reply-To: References: <4eef21f4-ef17-4c59-8c89-59a6a96c43cb@googlegroups.com> Message-ID: <088de60f-4248-3d5b-4559-0e300b3e26af@mrabarnett.plus.com> On 2018-02-15 11:25, Chris Warrick wrote: > On 15 February 2018 at 12:07, Sum J wrote: >> Below is my code. Here I want to read the "ip address" from s >> >> >> s= ''' >> Power On Enabled = On >> State: connected >> Radio Module: Unknown >> noise: -097 >> signalStrength: -046 >> ip address: 192.168.75.147 >> subnet mask: 255.255.255.0 >> IPv4 address configured by DHCP >> Mac Addr: ac:e2:d3:32:00:5a >> Mode: infrastrastructure >> ssid: Cloudlab >> Channel: 1 >> Regulatory: World Safe >> Authencation: WPA2/PSK >> Encryption: AES or TKIP >> ''' >> >> s = s.replace("=",":") >> # s = s.strip() >> print s >> >> d = {} >> for i in s: >> key, val = i.split(":") >> d[key] = val.strip() >> >> print d >> print d["ip address"] >> >> >> Getting below error : >> key, val = i.split(":") >> ValueError: need more than 1 value to unpack >> -- >> https://mail.python.org/mailman/listinfo/python-list > > If you iterate over a string, you are iterating over individual > characters. Instead, you need to split it into lines, first stripping > whitespace (starts and ends with an empty line). > > s = s.strip().replace("=",":") > print s > > d = {} I'd use .splitlines instead: > for i in s.split('\n'): > try: .partition works better here. You should note that the "Mac Addr" line has multiple colons, most of which are part of the value! > key, val = i.split(":") > d[key.strip()] = val.strip() > except ValueError: > print "no key:value pair found in", i > s = s.replace("=",":") d = {} for line in s.splitlines(): key, sep, val = line.partition(":") if sep: d[key.strip()] = val.strip() else: print "no key:value pair found in", line > > (PS. please switch to Python 3) > +1 From androidmaroso at gmail.com Thu Feb 15 17:30:24 2018 From: androidmaroso at gmail.com (Maroso Marco) Date: Thu, 15 Feb 2018 14:30:24 -0800 (PST) Subject: Suggestions on programming in Python an email simple client In-Reply-To: <050fb5ca-4ca7-4ffc-a8ff-6a68f645aae1@googlegroups.com> References: <050fb5ca-4ca7-4ffc-a8ff-6a68f645aae1@googlegroups.com> Message-ID: Il giorno marted? 13 febbraio 2018 21:06:19 UTC+1, Maroso Marco ha scritto: > Hi, > > what i'm trying to do is develop my own email client, but a simple one. > > I just want it to connect to a specific email account and read the subject line of messages coming from a certain email address. > > I then want it to be able to execute the command i wrote on the subject. > > It would be nice if i could specify two parameters on the subject line > > first : password; > second : command; > > The first parameter is the password and the second is the command to be executed in the local pc where this program is running. > > The program will have to show only email coming from a specific account > and if the mail is unread then it should read the subject line and find parameters. > The first parameter must match my hardcoded password, and the second parameter must be executed on the local pc (example ... c:\windows\notepad.exe) > > It must check for new emails at a given timing (example ...every 30 seconds ) > (the timing can be hardcoded or specified in a separate file) > > This is it. > > Any suggestions? > > I have started doing this (but i never programmed in python before): > > > import imaplib > import os > import email > import email.header > > plusmail = "emailaddresstobechecked at gmail.com" > googlepass = "mypassword" > owner = "validsender at gmail.com" > > M = imaplib.IMAP4_SSL('imap.gmail.com') > M.login(plusmail, googlepass) > M.select() > > status, response = M.search(None, '(UNSEEN)') > unread_msg_nums = response[0].split() > > # Print the count of all unread messages > print (len(unread_msg_nums)) > > # Print all unread messages from a certain sender of interest > status, response = M.search(None, '(UNSEEN)', '(FROM "%s")' % (owner)) > unread_msg_nums = response[0].split() > da = [] > for e_id in unread_msg_nums: > _, response = imap.fetch(e_id, '(BODY.PEEK[HEADER.FIELDS (From Subject)] RFC822.SIZE)') > da.append(response[0][1]) > # print (da) > > # Mark them as seen > for e_id in unread_msg_nums: > imap.store(e_id, '+FLAGS', '\Seen') > > > > typ, data = M.search(None, 'From',(owner)) > for num in data[0].split(): > typ, data = M.fetch(num, '(RFC822)') > print ('Message %s\n%s\n' % (num, data[0][1])) > M.close() > M.logout() > > I know there are duplicates but i need to figure out how to build this correctly > > > Thanks in advance for any help given. OK, i came up to this solution, but now i tried to compile and run as a service, but it doesnt seem to work, not fetching emails and reading comand. # A SCRIPT FROM MININGFELLOWS By CrazyLion import imaplib import os import email import email.header import time import subprocess def mailcontroller(): # Set user, pass and allowed mail for giving commands plusmail = "mailmailaddress.com" googlepass = "thepassword" captain = "autorizedmail at gmail.com" # Set vars for IMAP access M = imaplib.IMAP4_SSL('imap.gmail.com') M.login(plusmail, googlepass) M.select() # Set search on UNSEEN messages status, response = M.search(None, '(UNSEEN)') unread_msg_nums = response[0].split() # Mark as read for e_id in unread_msg_nums: M.store(e_id, '+FLAGS', '\Seen') # cycle messages sent from autorized email address typ, data = M.search(None, 'From',(captain)) for num in data[0].split(): typ, data = M.fetch(num, '(RFC822)') msg = email.message_from_string(data[0][1]) decode = email.header.decode_header(msg['Subject'])[0] subject = unicode(decode[0]) comando = subject if googlepass in subject: # print 'Message %s: %s' % (num, subject) # Split subject line googlepass,comando = subject.split(";") # Execute command #os.system(comando) # Execute command with alternate method subprocess.call(comando) # Delete email M.store(num, '+FLAGS', '\\Deleted') M.close() M.logout() # Read ini file for timer settings timing = open('timing.ini', 'r').read() # Convert timer value from string to int time.sleep(int(timing)) while True: mailcontroller() Any suggestions on how to compile it to run as a service. I've tried to compile it in 32 and 64 bit version. 32 bit version runs ok, the 64 shuts after some seconds. How to run comands like "net stop spooler" or "notepad" or "c:\file.bat" but as administrator (i mean with admin rights) Tried to create a service based on my compiled version, it is in the processes but it does not do anything. Thanks really. From michael.stemper at gmail.com Thu Feb 15 18:14:21 2018 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Thu, 15 Feb 2018 17:14:21 -0600 Subject: Suggestions on programming in Python an email simple client In-Reply-To: References: <050fb5ca-4ca7-4ffc-a8ff-6a68f645aae1@googlegroups.com> Message-ID: On 2018-02-15 16:30, Maroso Marco wrote: > Il giorno marted? 13 febbraio 2018 21:06:19 UTC+1, Maroso Marco ha scritto: >> Hi, >> >> what i'm trying to do is develop my own email client, but a simple one. >> >> I just want it to connect to a specific email account and read the subject line of messages coming from a certain email address. >> >> I then want it to be able to execute the command i wrote on the subject. >> >> It would be nice if i could specify two parameters on the subject line >> >> first : password; >> second : command; >> >> The first parameter is the password and the second is the command to be executed in the local pc where this program is running. You should never write anything that requires or even allows providing a password on a command line. On any multiuser machine, somebody can look for processes that have your program as the command and see the password. Passwords should only ever be provided interactively -- Michael F. Stemper Soglin for governor. From tkadm30 at yandex.com Fri Feb 16 06:22:04 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 16 Feb 2018 06:22:04 -0500 Subject: How to benchmark a HTTP connection with requests module? In-Reply-To: References: <2c77caf1-63db-da7d-865d-df94d0661d48@yandex.com> Message-ID: <8022211f-a7e2-8fd3-5bf4-af0bdfff2f1a@yandex.com> Hi Dennis, Nice pseudo code! :-) Is it possible benchmark/measure the latency of a HTTP connection for each threads by timing the duration of the requests.get method to complete? I also plan to test with gevent.monkey extension for enabling cooperative multithreading support: from gevent import monkey monkey.patch_all() Etienne Le 2018-02-15 ? 11:56, Dennis Lee Bieber a ?crit?: > > Keyword: threads > > Create a number of threads, each handling one request, and use a global > flag to start them. And maybe a queue to retrieve > > Pseudo-code (I haven't checked the manual for correct API): > > hold = True > resultQ = Queue.Queue() > > def worker(ID): > while hold: pass > r = requests.get(...) > resultQ.put( (ID, r.status_code) ) > > > def benchmark(): > requestTasks = [ threading.thread( worker, args=(ID) ) > for ID in range(NUMBEROFREQUESTS) ] > for rT in requestTasks.start() #or is it .run() > > #workers are now busy waiting for hold to False > #better would be to use threading.Condition and .notifyAll() > #having each thread wait on a global condition variable, > #rather than spinning on a busy wait > hold = False > > for _ in range(NUMBEROFREQUESTS): > (ID, code) = resultQ.get() > requestTasks[ID].join() > > #put any asserts here -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From dsujit2208 at gmail.com Fri Feb 16 11:04:21 2018 From: dsujit2208 at gmail.com (dsujit2208 at gmail.com) Date: Fri, 16 Feb 2018 08:04:21 -0800 (PST) Subject: Best site Message-ID: <92b2dfa6-151e-4c98-a879-f7837e78e3fa@googlegroups.com> which is the best site to learn python for data science. From tkadm30 at yandex.com Fri Feb 16 11:29:29 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 16 Feb 2018 11:29:29 -0500 Subject: How to benchmark a HTTP connection with requests module? In-Reply-To: <2hvd8d1e9vbq3ue4h23hebp2apkauht08p@4ax.com> References: <2c77caf1-63db-da7d-865d-df94d0661d48@yandex.com> <8022211f-a7e2-8fd3-5bf4-af0bdfff2f1a@yandex.com> <2hvd8d1e9vbq3ue4h23hebp2apkauht08p@4ax.com> Message-ID: <834c0c06-05f0-651f-9527-a59e77b1aa12@yandex.com> Hi Dennis, Here's my code so far: import os import requests import threading import time from Queue import Queue from test_support import unittest test_queue = Queue() def worker(ident, url, queue): ??? while True: ??????? start = time.clock() ??????? r = requests.get(url) ??????? end = time.clock() ??????? queue.put((ident, r.status_code, end - start)) ??????? queue.task_done() ??? return queue class HTTPBenchmarkTestCase(unittest.TestCase): ??? url = 'http://localhost/benchmark/' ??? threads = 5 ??? def setUp(self): ??????? self.hold = True ??? def test_benchmark_concurrency(self): ??????? for i in range(self.threads): ??????????? t = threading.Thread(target=worker, args=(i, self.url, test_queue)) ??????????? t.daemon = True ??????????? t.start() ??????????? obj = test_queue.get() ??????????? print obj ??????? test_queue.join() erob at marina:~/src/django-hotsauce-0.9/tests$ pypy ./run.py -C benchmarks DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (0, 200, 0.12999503499999943) DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (1, 200, 0.053242928999999606) DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (0, 200, 0.11221248100000025) DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (1, 200, 0.12075822500000033) DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost DEBUG:http://localhost:80 "GET /benchmark/ HTTP/1.1" 200 76 DEBUG:Starting new HTTP connection (1): localhost (0, 200, 0.12590276600000028) . ---------------------------------------------------------------------- Ran 1 test in 0.468s OK What do you think? :-) Etienne Le 2018-02-16 ? 10:58, Dennis Lee Bieber a ?crit?: > On Fri, 16 Feb 2018 06:22:04 -0500, Etienne Robillard > declaimed the following: > >> Hi Dennis, >> >> Nice pseudo code! :-) >> >> Is it possible benchmark/measure the latency of a HTTP connection for >> each threads by timing the duration of the requests.get method to complete? >> > Wrap each .get with calls for start time and end time? (Need to check > if time.time or time.clock is more precise on your hardware, docs recommend > time.clock for benchmarking) > >>> def worker(ID): >>> while hold: pass > st = time.clock() >>> r = requests.get(...) > en = time.clock() > resultQ.put( (ID, r.status_code, en - st) ) > #return duration as third parameter > > >>> for _ in range(NUMBEROFREQUESTS): > (ID, code, duration) = resultQ.get() >>> requestTasks[ID].join() -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From none at invalid.com Fri Feb 16 12:53:31 2018 From: none at invalid.com (mm0fmf) Date: Fri, 16 Feb 2018 17:53:31 +0000 Subject: Best site In-Reply-To: <92b2dfa6-151e-4c98-a879-f7837e78e3fa@googlegroups.com> References: <92b2dfa6-151e-4c98-a879-f7837e78e3fa@googlegroups.com> Message-ID: On 16/02/2018 16:04, dsujit2208 at gmail.com wrote: > which is the best site to learn python for data science. > Google. From irving.duran at gmail.com Fri Feb 16 15:07:09 2018 From: irving.duran at gmail.com (Irving Duran) Date: Fri, 16 Feb 2018 14:07:09 -0600 Subject: Best site In-Reply-To: References: <92b2dfa6-151e-4c98-a879-f7837e78e3fa@googlegroups.com> Message-ID: <19003d0d-6e36-2a8e-7a16-0518a890495d@gmail.com> Besides Google for sure. I liked www.codecademy.com Thank you, Irving Duran On 02/16/2018 11:53 AM, mm0fmf wrote: > On 16/02/2018 16:04, dsujit2208 at gmail.com wrote: >> which is the best site to learn python for data science. >> > > Google. From none at invalid.com Fri Feb 16 15:52:19 2018 From: none at invalid.com (mm0fmf) Date: Fri, 16 Feb 2018 20:52:19 +0000 Subject: Best site In-Reply-To: References: <92b2dfa6-151e-4c98-a879-f7837e78e3fa@googlegroups.com> <19003d0d-6e36-2a8e-7a16-0518a890495d@gmail.com> Message-ID: A. Because it spoils the flow. On 16/02/2018 20:07, Irving Duran wrote: > Besides Google for sure. I liked www.codecademy.com > > Thank you, > > Irving Duran > > On 02/16/2018 11:53 AM, mm0fmf wrote: >> On 16/02/2018 16:04, dsujit2208 at gmail.com wrote: >>> which is the best site to learn python for data science. >>> >> >> Google. > Q. Why shouldn't I top post on a newsgroup? From aewindhorn at gmail.com Fri Feb 16 17:18:16 2018 From: aewindhorn at gmail.com (windhorn) Date: Fri, 16 Feb 2018 14:18:16 -0800 (PST) Subject: How to run script from interpreter? Message-ID: Yes, it's been covered, but not quite to my satisfaction. Here's an example simple script: # Very simple script bar = 123 I save this as "foo.py" somewhere Python can find it >>> import foo >>> bar Traceback (most recent call last): File "", line 1, in NameError: name 'bar' is not defined # Oops, it's in a different namespace and I have to prefix EVERYTHING with "foo.". This is inconvenient. >>> foo.bar 123 Start a new session... >>> from foo import * >>> bar 123 Do a little editing: # Very simple (new) script bar = 456 Save as foo.py, back to the interpreter: >>> reload(foo) Traceback (most recent call last): File "", line 1, in NameError: name 'foo' is not defined # Oops, it's not in the namespace, so there is NO way to use reload Start a new session... >>> execfile('foo.py') Traceback (most recent call last): File "", line 1, in IOError: [Errno 2] No such file or directory: 'foo.py' # Oops, it's on the path, but execfile can't find it >>> import os,sys >>> os.chdir('W:/Code') >>> execfile('foo.py') >>> bar 456 Do some more editing: # Very simple (even newer) script bar = 789 Save it as foo.py, back to the interpreter: >>> execfile('foo.py') >>> bar 789 That works, but nothing is very convenient for debugging simple scripts. If I run the script from a command prompt it works, but I lose all my other stuff (debugging functions, variables, etc.). More a comment than a question but seems like sometimes execfile() is the right tool. Regards, Allen From rosuav at gmail.com Fri Feb 16 17:34:34 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 09:34:34 +1100 Subject: How to run script from interpreter? In-Reply-To: References: Message-ID: On Sat, Feb 17, 2018 at 9:18 AM, windhorn wrote: > Yes, it's been covered, but not quite to my satisfaction. > > Here's an example simple script: > > # Very simple script > bar = 123 > > I save this as "foo.py" somewhere Python can find it > >>>> import foo >>>> bar > Traceback (most recent call last): > File "", line 1, in > NameError: name 'bar' is not defined > > # Oops, it's in a different namespace and I have to prefix EVERYTHING with "foo.". This is inconvenient. > >>>> foo.bar > 123 > > Start a new session... > >>>> from foo import * >>>> bar > 123 > > Do a little editing: > > # Very simple (new) script > bar = 456 > > Save as foo.py, back to the interpreter: > >>>> reload(foo) > Traceback (most recent call last): > File "", line 1, in > NameError: name 'foo' is not defined > > # Oops, it's not in the namespace, so there is NO way to use reload > > Start a new session... > >>>> execfile('foo.py') > Traceback (most recent call last): > File "", line 1, in > IOError: [Errno 2] No such file or directory: 'foo.py' > > # Oops, it's on the path, but execfile can't find it > >>>> import os,sys >>>> os.chdir('W:/Code') >>>> execfile('foo.py') >>>> bar > 456 > > Do some more editing: > > # Very simple (even newer) script > bar = 789 > > Save it as foo.py, back to the interpreter: > >>>> execfile('foo.py') >>>> bar > 789 > > That works, but nothing is very convenient for debugging simple scripts. If I run the script from a command prompt it works, but I lose all my other stuff (debugging functions, variables, etc.). > > More a comment than a question but seems like sometimes execfile() is the right tool. Or possibly you want to exit completely, and run "python3 -i foo.py", which will drop you into the interactive interpreter after running the script. That's safer than trying to re-execute an already-loaded module. ChrisA From ian.g.kelly at gmail.com Fri Feb 16 17:47:23 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 16 Feb 2018 15:47:23 -0700 Subject: How to run script from interpreter? In-Reply-To: References: Message-ID: On Fri, Feb 16, 2018 at 3:18 PM, windhorn wrote: > Yes, it's been covered, but not quite to my satisfaction. > > Here's an example simple script: > > # Very simple script > bar = 123 > > I save this as "foo.py" somewhere Python can find it > >>>> import foo >>>> bar > Traceback (most recent call last): > File "", line 1, in > NameError: name 'bar' is not defined > > # Oops, it's in a different namespace and I have to prefix EVERYTHING with "foo.". This is inconvenient. > >>>> foo.bar > 123 > > Start a new session... > >>>> from foo import * >>>> bar > 123 You didn't need to start a new session here. "from foo import *" works just as well in the existing session. > Do a little editing: > > # Very simple (new) script > bar = 456 > > Save as foo.py, back to the interpreter: > >>>> reload(foo) > Traceback (most recent call last): > File "", line 1, in > NameError: name 'foo' is not defined > > # Oops, it's not in the namespace, so there is NO way to use reload Sure there is. py> import foo # Get the module to reload py> reload(foo) # Reload it py> from foo import * # Update all the names previously *-imported > Start a new session... > >>>> execfile('foo.py') > Traceback (most recent call last): > File "", line 1, in > IOError: [Errno 2] No such file or directory: 'foo.py' > > # Oops, it's on the path, but execfile can't find it execfile doesn't use the path; that's not what it's meant for. From robertvstepp at gmail.com Fri Feb 16 22:22:48 2018 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 16 Feb 2018 21:22:48 -0600 Subject: Are the critiques in "All the things I hate about Python" valid? Message-ID: This article is written by Nathan Murthy, a staff software engineer at Tesla. The article is found at: https://medium.com/@natemurthy/all-the-things-i-hate-about-python-5c5ff5fda95e Apparently he chose his article title as "click bait". Apparently he does not really hate Python (So he says.). His leader paragraph is: "Python is viewed as a ubiquitous programming language; however, its design limits its potential as a reliable and high performance systems language. Unfortunately, not every developer is aware of its limitations." As I currently do not have the necessary technical knowledge to properly evaluate his claims, I thought I would ask those of you who do. I have neither the knowledge or boB-hours to write a large distributed system code base, but I am curious if Python is truly limited for doing these in the ways he claims. BTW, I am not trying to start (another) heated, emotional thread. You guys do sometimes get carried away! I honestly just what to know the truth of the matters out of my continuing to learn Python. I suspect there is probably some truth in his claims, but I am not sure if he is taking things out of their proper application contexts or not. Thanks! -- boB From ben+python at benfinney.id.au Fri Feb 16 22:49:38 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 17 Feb 2018 14:49:38 +1100 Subject: Click bait versus =?utf-8?Q?=E2=80=9Crational?= and unbiased =?utf-8?Q?demeanor=E2=80=9D?= (was: Are the critiques in "All the things I hate about Python" valid?) References: Message-ID: <854lmgcm25.fsf@benfinney.id.au> boB Stepp writes: > Apparently he chose his article title as "click bait". Apparently he > does not really hate Python (So he says.). Those may well be true. What I find implausible is his expressed desire: Ok, so ?hate? is a strong word, but hopefully this click-baits enough folks into reading this article with a clearer, more rational and unbiased demeanor than what the title suggests. Without descending into hyperbole, what I hope to share with you are all [?] At this point, having deliberately chosen a hyperbolic headline, the author is too late to avoid hyperbole, and too late to engender clear, rational unbiased demeanour. I despise the tendency to do all the things designed explicitly to push our irrational buttons, and then attempt feebly to regain the moral high ground by implying that the *reader* is responsible for any lack of ?clear, rational, unbiased demeanour?. The author is not writing in good faith, and I do not expect that any arguments found in it are honest either. -- \ ?I find the whole business of religion profoundly interesting. | `\ But it does mystify me that otherwise intelligent people take | _o__) it seriously.? ?Douglas Adams | Ben Finney From BILL_NOSPAM at Noway.net Fri Feb 16 22:50:23 2018 From: BILL_NOSPAM at Noway.net (Bill) Date: Fri, 16 Feb 2018 22:50:23 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: boB Stepp wrote: > This article is written by Nathan Murthy, a staff software engineer at > Tesla. The article is found at: > https://medium.com/@natemurthy/all-the-things-i-hate-about-python-5c5ff5fda95e > > Apparently he chose his article title as "click bait". Apparently he > does not really hate Python (So he says.). His leader paragraph is: > > "Python is viewed as a ubiquitous programming language; however, its > design limits its potential as a reliable and high performance systems > language. Python is simply not a high performance system language, but it has other virtues. You would not likely choose Python for a sophisticated distributed computing application, but not everyone needs to write a sophisticated distributed computing application. I believe those that do, will be conscientious enough to choose an appropriate tool. > Unfortunately, not every developer is aware of its > limitations." > > As I currently do not have the necessary technical knowledge to > properly evaluate his claims, I thought I would ask those of you who > do. I have neither the knowledge or boB-hours to write a large > distributed system code base, but I am curious if Python is truly > limited for doing these in the ways he claims. > > BTW, I am not trying to start (another) heated, emotional thread. You > guys do sometimes get carried away! I honestly just what to know the > truth of the matters out of my continuing to learn Python. I suspect > there is probably some truth in his claims, but I am not sure if he is > taking things out of their proper application contexts or not. > > Thanks! From ben+python at benfinney.id.au Fri Feb 16 23:05:34 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 17 Feb 2018 15:05:34 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? References: Message-ID: <85zi48b6r5.fsf@benfinney.id.au> boB Stepp writes: > https://medium.com/@natemurthy/all-the-things-i-hate-about-python-5c5ff5fda95e > As I currently do not have the necessary technical knowledge to > properly evaluate his claims, I thought I would ask those of you who > do. Thank you for asking. The author makes many mistakes. He blithely conflates ?weakly typed? (Python objects are not weakly, but very strongly typed) with ?dynamically typed? (yes, Python's name binding is dynamically typed). Those are two, orthognal dimensions to describe a language. All the assertions about ?strongly typed? should instead be read as ?statically typed?. I do not think the author understands the differences, nor the trade-offs involved, in the weak?strong dimension nor the static?dynamic dimension. The author conflates the Python language, which has many different implementations and interpreters on very different technologies, with one particular implementation (CPython). Python does not have fast or strong performance; the performance is entirely a property of a specific implementation. All the arguments about performance should be read as criticisms of whatever implementation the author is using. He doesn't say which, so we can't know how justified those complaints are. A strong hint is in the complaint about the Global Interpreter Lock. This is a property of only one implementation: the CPython implementation. It is a known limitation and many people have worked for many years to ameliorate it; it is unclear whether the author knows of anything done in recent years (the only specific information cited is for 2012; a reminder, that's *six years ago*, a aeon in platform development). The author also uses very old information when complaining about Python 2 versus Python 3. The citation of grave fears is to a 2014 article. The author shows no signs of awareness that the Python 3 ecosystem is very healthy today compared to Python 2. The complaint about abstractions makes another silly conflation; the author seems to think any language that does object-orientation differently from Java, is obviously wrong. The author may benefit from reading (advice which is no less relevant than in 2004). But maybe not. The complaint about Python's build system and packaging tools is accurate enough, though pretty parochial (the author has a strong bias to existing Fortran packages). The Python build and packaging ecosystem has been improving steadily for many years now, but yes, it still needs a lot of improvement. The author acknowledges in the conclusion that ?nothing I?ve said is new?. I would argue that most of what is said is not even true; the article should not be referenced at all, IMO. -- \ ?We jealously reserve the right to be mistaken in our view of | `\ what exists, given that theories often change under pressure | _o__) from further investigation.? ?Thomas W. Clark, 2009 | Ben Finney From python at mrabarnett.plus.com Fri Feb 16 23:07:51 2018 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 17 Feb 2018 04:07:51 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 2018-02-17 03:22, boB Stepp wrote: > This article is written by Nathan Murthy, a staff software engineer at > Tesla. The article is found at: > https://medium.com/@natemurthy/all-the-things-i-hate-about-python-5c5ff5fda95e > > Apparently he chose his article title as "click bait". Apparently he > does not really hate Python (So he says.). His leader paragraph is: > > "Python is viewed as a ubiquitous programming language; however, its > design limits its potential as a reliable and high performance systems > language. Unfortunately, not every developer is aware of its > limitations." > > As I currently do not have the necessary technical knowledge to > properly evaluate his claims, I thought I would ask those of you who > do. I have neither the knowledge or boB-hours to write a large > distributed system code base, but I am curious if Python is truly > limited for doing these in the ways he claims. > > BTW, I am not trying to start (another) heated, emotional thread. You > guys do sometimes get carried away! I honestly just what to know the > truth of the matters out of my continuing to learn Python. I suspect > there is probably some truth in his claims, but I am not sure if he is > taking things out of their proper application contexts or not. > He's making the assertions that: "dynamically typed" == "weakly typed" and: "strongly typed" == "statically typed" They aren't equivalences. Python is both dynamically typed _and_ strongly typed. And static type checking gets you only so far. If you've added when you should've subtracted, static type checking won't spot it. Another point: a "wheel" is not a "source distribution" that requires a compiler, it can include a pre-compiled binary. At least in the Conclusion he says "It is still a useful and powerful programming language when suited for its intended purposes." Exactly. It's not suited for realtime processing, etc, but, then, it was never intended for that anyway. From rosuav at gmail.com Fri Feb 16 23:25:15 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 15:25:15 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sat, Feb 17, 2018 at 2:22 PM, boB Stepp wrote: > This article is written by Nathan Murthy, a staff software engineer at > Tesla. The article is found at: > https://medium.com/@natemurthy/all-the-things-i-hate-about-python-5c5ff5fda95e > > Apparently he chose his article title as "click bait". Apparently he > does not really hate Python (So he says.). His leader paragraph is: > > "Python is viewed as a ubiquitous programming language; however, its > design limits its potential as a reliable and high performance systems > language. Unfortunately, not every developer is aware of its > limitations." > > As I currently do not have the necessary technical knowledge to > properly evaluate his claims, I thought I would ask those of you who > do. I have neither the knowledge or boB-hours to write a large > distributed system code base, but I am curious if Python is truly > limited for doing these in the ways he claims. > > BTW, I am not trying to start (another) heated, emotional thread. You > guys do sometimes get carried away! I honestly just what to know the > truth of the matters out of my continuing to learn Python. I suspect > there is probably some truth in his claims, but I am not sure if he is > taking things out of their proper application contexts or not. 1) Type safety. This is often touted as a necessity for industrial-grade software. It isn't. There are many things that a type system, no matter how sophisticated, cannot catch; for some reason, though, we don't hear people saying "C is useless for industrial-grade software because it doesn't have function contracts". Anyway, if you want some system of type checking, you can use static analysis (eg tools like MyPy) to go over your code the same way a compiler might. "The first glaring issue is that I have no guarantee that is_valid() returns a bool type." -- huh? It's being used in a boolean context, and it has a name that starts "is_". How much guarantee are you looking for? *ANY* object can be used in an 'if', so it doesn't even matter. This is a stupidly contrived criticism. > Python markets itself as a dynamically-typed programming language, > similar to Perl, Ruby, or JavaScript. The word ?dynamically typed? has > a positive connotation, more so than ?weakly typed.? Conversely, the > word ?strongly typed? sounds more positive than saying ?statically typed.? Those are all different terms, and all of them are misunderstood. They're not synonyms. > Python ships with a threading module; but due to how the Python > interpreter is implemented, it can only ever run one thread at a time. > This is due to the infamous Global Interpreter Lock (GIL). In an age > when the only computers around had a single core, this was > nothing to fuss over. Totally not true. The GIL does not stop other threads from running. Also, Python has existed for multiple CPU systems pretty much since its inception, I believe. (Summoning the D'Aprano for history lesson?) > Python is relatively slow compared to programming languages that > run closer to the operating system. The run time of the countdown > example above is orders of magnitude faster when implemented > in other language runtimes. Yeah, it sometimes runs a bit slower in benchmarking. And it's way WAY faster to code in. Case in point: I reimplemented an app in Python based on a C# (one of the languages the article's author touts as better than Python), and made a single file under a thousand lines long out of what had been two separate projects (a UI and a library), dozens of files, tens of thousands of lines, and tons of boilerplate that looks like this: https://github.com/geel9/SteamAuth/blob/master/SteamAuth/SteamGuardAccount.cs#L15 That's thirty lines of declarations to show how the JSON data translates into this object's properties. In Python, I just decode JSON to a dict, and then subscript that dict with whatever I need. Zero lines of boilerplate. So which one is more expensive? Hours and hours of programmer time, or a few milliseconds of runtime? > If you?re doing development on Mac or Ubuntu, you likely already > have Python 2.7 installed (Windows normally does not come with > Python pre-packaged). There?s a good chance that when you first > started programming in Python, no one told you about this thing > called Python 3 (no one told me at least). Ubuntu ships with Python 3 installed these days (and they're working on removing the need to have Python 2 installed). Macs ship with a lot of outdated software, so the correct thing to do is to install the latest anyway. If "no one told you" about Python 3, you don't know how to ask. And trust me, the Py2/Py3 gap is far less serious than the differences between version X and version X+1 of many JavaScript libraries. I know this for certain, having spent this past week resolving problems with React and related package. > Encapsulation and Packaging > ... > This is unsettling if you?re coming from a Java or C/C++ world and > are expecting access modifiers. They simply do not exist in Python. Yeah, it takes some getting used to. (Note that C doesn't have access modifiers either, so it's really Java and C++.) Did you know, though, that C++ is quite happy to let you cast a pointer to another type, and then ignore all the access rules? Yeah, but that doesn't bother people. Ultimately, you can always break past those kinds of protections. So why not put the protection into your source code instead of the compiler? The leading-underscore convention makes it obvious any time you "reach in" and manipulate private members. Sure, nothing stops you from doing it. But that's why we read code, we don't just run it. Yep, it's yet another hyperbole-filled article worded as click-bait. Fortunately it does have some content in it, but most of it is the same old complaints that have been known about - and solved - for years. ChrisA From rosuav at gmail.com Fri Feb 16 23:32:37 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 15:32:37 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sat, Feb 17, 2018 at 2:50 PM, Bill wrote: > boB Stepp wrote: >> >> This article is written by Nathan Murthy, a staff software engineer at >> Tesla. The article is found at: >> >> https://medium.com/@natemurthy/all-the-things-i-hate-about-python-5c5ff5fda95e >> >> Apparently he chose his article title as "click bait". Apparently he >> does not really hate Python (So he says.). His leader paragraph is: >> >> "Python is viewed as a ubiquitous programming language; however, its >> design limits its potential as a reliable and high performance systems >> language. > > > Python is simply not a high performance system language, but it has other > virtues. You would not likely choose Python for a sophisticated distributed > computing application, but not everyone needs to write a sophisticated > distributed computing application. I believe those that do, will be > conscientious enough to choose an appropriate tool. > You'd be surprised how rarely that kind of performance even matters. The author of that article cites C# as a superior language, but in the rewrite from C# to Python (the same one I mentioned in the other post), I sped the program up incredibly. Part of that is because C# requires the startup of its framework (in my case that's Mono) just as Python does, and partly it's because the simplicity of Python let me eliminate a number of unnecessary HTTP requests. Trust me, your choice of language doesn't help you if it means you do three (sequential) HTTP requests when one would have done. Clean code has its own advantages. And if the number of HTTP transactions didn't change, the performance of the two languages - or any other languages you choose - would be virtually indistinguishable. Most programs spend most of their time waiting, perhaps on the network, or the disk, or the user, but always waiting. ChrisA From steve+comp.lang.python at pearwood.info Fri Feb 16 23:36:05 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 17 Feb 2018 04:36:05 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: Message-ID: On Fri, 16 Feb 2018 21:22:48 -0600, boB Stepp wrote: > This article is written by Nathan Murthy, a staff software engineer at > Tesla. The article is found at: > https://medium.com/@natemurthy/all-the-things-i-hate-about- python-5c5ff5fda95e > > Apparently he chose his article title as "click bait". Apparently he > does not really hate Python (So he says.). His leader paragraph is: > > "Python is viewed as a ubiquitous programming language; however, its > design limits its potential as a reliable and high performance systems > language. Unfortunately, not every developer is aware of its > limitations." I haven't (yet) read the article, but the above seems reasonable to me. Python's design does put rather harsh limits on how far you can push the compiler to generate high-performance code. I'm not so sure about the claim for "reliable" -- I suspect that's actually *not* a reasonable conclusion to draw (but I shall keep an open mind until I've read the article). If high-performance is important to (generic) you, the right way to use Python is: - use Python for the parts of the application which are not performance critical, e.g. the user interface; - prototype the rest of the application in Python as a first iteration; - profile the application to find the *actual* bottlenecks, not the parts of the code you *imagined* would be bottlenecks; - re-write those parts in a higher performance language; - and use Python as the glue to hold those pieces together. This is why Python is prodding buttock in the field of scientific computing. That's precisely how numpy works: the back-end is written in C and Fortran for speed, and the front-end user-interface and non-critical code is written in Python. Python is really good for gluing together high-performance but user- and programmer-hostile scientific libraries written in C and Fortran. You wouldn't write a serious, industrial-strength neural network in pure Python code and expect to process terabytes of data in any reasonable time. But you might write a serious, industrial-strength neural network in C or Rust, and then use your Python layer as the front-end to it, feeding data in and out of the neural network from easy-to-write, easy-to-debug, easy- to-maintain Python code. And you can write a prototype neural network in Python. It won't process gigabytes of data per second, but it will probably process megabytes. And with tools like cython, you can often turn your Python code into Python- like code that is compiled into efficient machine code, giving you almost C-like speed with almost Python-like simplicity. Remember that Python's original purpose was to act as a minimalist glue language between C and Fortran libraries. Over the years Python has developed enough features and power to be far more than that: its an excellent generalist, a high-performance scripting language and low to medium performance application language[1]. But acting as a glue between components written in other languages is still a very powerful technique for Python programmers. The standard library works like this too: non-critical libraries, like those for manipulating command line arguments, are written in pure Python. Critical components, like dicts and sets, are written in C. And then there are libraries that were prototyped in Python, then re-written in C for speed once they were mature, like the decimal and fractions modules. [1] Pedants will (rightly) observe that computer languages aren't themselves high- or low-performance. Languages are abstract entities that cannot be described as "fast" or "slow", as they run at the speed of imagination. Only specific interpreters/compilers can be described as fast or slow. Here I am talking about the standard CPython interpreter. There are five mainstream, mature Python interpreters: CPython, Jython, IronPython, PyPy and Stackless. They all have their individual strengths and weaknesses. -- Steve From robertvstepp at gmail.com Fri Feb 16 23:54:12 2018 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 16 Feb 2018 22:54:12 -0600 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <85zi48b6r5.fsf@benfinney.id.au> References: <85zi48b6r5.fsf@benfinney.id.au> Message-ID: On Fri, Feb 16, 2018 at 10:05 PM, Ben Finney wrote: > He blithely conflates ?weakly typed? (Python objects are not weakly, but > very strongly typed) with ?dynamically typed? (yes, Python's name > binding is dynamically typed). Those are two, orthognal dimensions to > describe a language. > > All the assertions about ?strongly typed? should instead be read as > ?statically typed?. I do not think the author understands the > differences, nor the trade-offs involved, in the weak?strong dimension > nor the static?dynamic dimension. I should have caught the "weakly typed" error. But I was too distracted by his "type safety" train of thought. And this is one I am still puzzling over: Are statically-typed languages inherently "safer" than properly implemented dynamically-typed languages? I can see the advantages of catching type errors at compile time versus run time. And with the relatively recent introduction of type hints to python, I was wondering if this was indeed a perceived, need-to-correct weakness by the python devs? But I have seen myself the advantages of being able to rebind identifiers to differently typed objects than what they started out being bound to. So I do not have enough knowledge to properly evaluate software safety or robustness in terms of how a language is typed. > A strong hint is in the complaint about the Global Interpreter Lock. > This is a property of only one implementation: the CPython > implementation. It is a known limitation and many people have worked for > many years to ameliorate it; it is unclear whether the author knows of > anything done in recent years (the only specific information cited is > for 2012; a reminder, that's *six years ago*, a aeon in platform > development). This is another of my big technical weak spots. I currently believe that doing concurrent and asynchronous programming is a difficult chore in any language, but I have yet gotten to the point of where I know enough to start struggling myself with these forms of programming. I do know from some of my books that there are ways despite the GIL to handle these types of programming problems with the current CPython implementation. But I do not know enough to evaluate how well or poorly python can make use of multicore processors. > The author also uses very old information when complaining about Python > 2 versus Python 3. The citation of grave fears is to a 2014 article. The > author shows no signs of awareness that the Python 3 ecosystem is very > healthy today compared to Python 2. I mostly dismissed out of hand this section. The only disturbing part, in my mind, was the claim that individual companies have python 2 vs. 3 rifts within their companies. I hope that is not really true. > The complaint about abstractions makes another silly conflation; the > author seems to think any language that does object-orientation > differently from Java, is obviously wrong. The author may benefit from > reading > (advice which is no less relevant than in 2004). But maybe not. Again, my only concern here was software safety issue claims. Does locking down object attribute access by the language necessarily improve software safety or robustness? I actually like python's "we're all consenting adults" mentality. It seems to me that the quality of the code is more reflective of the quality of the programmers than protecting (mediocre?) programmers from themselves by language design. But then again, maybe these sorts of language design restrictions make for fewer errors, even by excellent programmers, than not doing so. Again, I don't believe I know enough to intelligently evaluate the pros and cons. > The author acknowledges in the conclusion that ?nothing I?ve said is > new?. I would argue that most of what is said is not even true; the > article should not be referenced at all, IMO. > Ben, I guess I am trying to be more charitable in my interpretation of the author's intent and actual knowledge. I was led to this article via one of the two weekly python news collation email services that I subscribe to, so I am hoping to learn something from the article and everyone's feedback. What's the old saw? Even a blind squirrel finds a nut once in a while? Thanks! -- boB From rosuav at gmail.com Fri Feb 16 23:54:38 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 15:54:38 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sat, Feb 17, 2018 at 3:36 PM, Steven D'Aprano wrote: > Python is really good for gluing together high-performance but user- and > programmer-hostile scientific libraries written in C and Fortran. You > wouldn't write a serious, industrial-strength neural network in pure > Python code and expect to process terabytes of data in any reasonable > time. > > But you might write a serious, industrial-strength neural network in C or > Rust, and then use your Python layer as the front-end to it, feeding data > in and out of the neural network from easy-to-write, easy-to-debug, easy- > to-maintain Python code. Easy-to-debug. This deserves some expansion. One of the best aspects of Python is its debuggability (from readable code, lots of introspection, etc), so when you convert low level functionality into C, you ideally want to start with the parts that can be most easily unit-tested. User interfaces are notoriously hard to unit test, but pure functions can be more thoroughly tested. So you translate the pure functions first. There's often a strong correlation between the easily-testable functions and the performance bottlenecks. I'm hand-waving away a ton of details and complexities here, and it's certainly not a hard-and-fast rule, but it's a very common correlation. ChrisA From robertvstepp at gmail.com Sat Feb 17 00:11:35 2018 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 16 Feb 2018 23:11:35 -0600 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Fri, Feb 16, 2018 at 10:25 PM, Chris Angelico wrote: > > 1) Type safety. > > This is often touted as a necessity for industrial-grade software. It > isn't... Chris, would you mind expanding on this point? What is necessary for industrial-grade, safe, robust software? Do statically-typed languages give any real advantage over dynamically-typed languages in this context? I know much is achieved by how the software is designed and the practices used to implement the design that are independent of the language used. But what language features/components are essential for industrial-grade software? -- boB From best_lay at yahoo.com Sat Feb 17 00:15:25 2018 From: best_lay at yahoo.com (Wildman) Date: Fri, 16 Feb 2018 23:15:25 -0600 Subject: Python 2 to 3 Conversion Message-ID: I have a bit of code I found on the web that will return the ip address of the named network interface. The code is for Python 2 and it runs fine. But, I want to use the code with Python 3. Below is the code followed by the error message. Suggestions appreciated. #!/usr/bin/env python3 import socket import fcntl import struct def get_ip_address(ifname): s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) return socket.inet_ntoa(fcntl.ioctl( s.fileno(), 0x8915, # SIOCGIFADDR struct.pack('256s', ifname[:15]) )[20:24]) print(get_ip_address("eth0")) print(get_ip_address("lo")) Traceback (most recent call last): File "./test.py", line 14, in print(get_ip_address("eth0")) File "./test.py", line 12, in get_ip_address struct.pack('256s', ifname[:15]) struct.error: argument for 's' must be a bytes object -- GNU/Linux user #557453 The cow died so I don't need your bull! From rosuav at gmail.com Sat Feb 17 00:16:24 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 16:16:24 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> Message-ID: On Sat, Feb 17, 2018 at 3:54 PM, boB Stepp wrote: > On Fri, Feb 16, 2018 at 10:05 PM, Ben Finney wrote: > >> He blithely conflates ?weakly typed? (Python objects are not weakly, but >> very strongly typed) with ?dynamically typed? (yes, Python's name >> binding is dynamically typed). Those are two, orthognal dimensions to >> describe a language. >> >> All the assertions about ?strongly typed? should instead be read as >> ?statically typed?. I do not think the author understands the >> differences, nor the trade-offs involved, in the weak?strong dimension >> nor the static?dynamic dimension. > > I should have caught the "weakly typed" error. But I was too > distracted by his "type safety" train of thought. And this is one I > am still puzzling over: Are statically-typed languages inherently > "safer" than properly implemented dynamically-typed languages? I can > see the advantages of catching type errors at compile time versus run > time. And with the relatively recent introduction of type hints to > python, I was wondering if this was indeed a perceived, > need-to-correct weakness by the python devs? But I have seen myself > the advantages of being able to rebind identifiers to differently > typed objects than what they started out being bound to. So I do not > have enough knowledge to properly evaluate software safety or > robustness in terms of how a language is typed. No, they're not inherently safer. There's no way for the language to guarantee that your code is bug-free. But we all know that all code has bugs in it - and that the sooner you find a bug, the easier it is to fix it. So we have a variety of tools that help us to detect bugs: * Syntax highlighting * Linters (eg "unused local variable") * Warnings of various forms * Type checkers * Naming conventions (eg using plurals for collections) * Clear and consistent indentation * Unit tests * Automated test running as part of CI/CD * Machine-readable documentation (eg 'assert') * Etcetera Each of them helps you to catch bugs before they get out into production. None of them makes one language "safer" than another. They're all tools that can help you, but sometimes one tool's benefit overlaps another's to the extent that it's not worth the effort. That's why most of them are optional. If type checking is of value to you, use it! If you think it's a pointless waste of time, ignore it! It's not hurting you. >> A strong hint is in the complaint about the Global Interpreter Lock. >> This is a property of only one implementation: the CPython >> implementation. It is a known limitation and many people have worked for >> many years to ameliorate it; it is unclear whether the author knows of >> anything done in recent years (the only specific information cited is >> for 2012; a reminder, that's *six years ago*, a aeon in platform >> development). > > This is another of my big technical weak spots. I currently believe > that doing concurrent and asynchronous programming is a difficult > chore in any language, but I have yet gotten to the point of where I > know enough to start struggling myself with these forms of > programming. I do know from some of my books that there are ways > despite the GIL to handle these types of programming problems with the > current CPython implementation. But I do not know enough to evaluate > how well or poorly python can make use of multicore processors. Asynchronicity and concurrency are hard. Getting your head around a program that is simultaneously doing two things is inherently tricky. Anything that simplifies this (such as providing atomic and thread-safe operations on high-level data structures) will make this sort of work FAR easier. That's what Python gives you. CPython's current implementation (with the GIL) is reasonably easy to conceptualize, and requires very few other locks to make everything work sanely; all attempts to remove the GIL from CPython have involved other tradeoffs, usually resulting in significant performance penalties on single-threaded code. GIL or no GIL, CPython *does* support threads, and they are incredibly useful. They just don't automatically allow you to use multiple CPU cores. That's all. ChrisA From rosuav at gmail.com Sat Feb 17 00:28:11 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 16:28:11 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sat, Feb 17, 2018 at 4:11 PM, boB Stepp wrote: > On Fri, Feb 16, 2018 at 10:25 PM, Chris Angelico wrote: > >> >> 1) Type safety. >> >> This is often touted as a necessity for industrial-grade software. It >> isn't... > > Chris, would you mind expanding on this point? What is necessary for > industrial-grade, safe, robust software? Do statically-typed > languages give any real advantage over dynamically-typed languages in > this context? I know much is achieved by how the software is designed > and the practices used to implement the design that are independent of > the language used. But what language features/components are > essential for industrial-grade software? > I've expanded on it a bit in my other post (which crossed in the mail with this of yours). Basically, type checking ("type safety" is a bit of a misnomer anyway) is just one of many possible ways to potentially catch bugs. So it's not a *necessity*, any more than correct indentation is a necessity. Both of them are valuable; neither of them is crucial. And code review can do what the language doesn't. Would you accept this JavaScript function in code review? function calculateSpam(x) { if (x>5) { return "spaaaaaaaaaam"; } else { return "spam"; }} The indentation doesn't matter, right? Not in JS. But it does to humans. Same goes for sane use of data types: def is_valid(x): if yada_yada: return "yes" return [x] Again, Python doesn't care if a function (a) sometimes returns a string and sometimes a list, and (b) always returns a truthy value, despite having a name that suggests a boolean return. But code review should catch this. Once you identify a common problem, you can get the computer to catch it for you, so it doesn't come up in code review all the time. Does that mean type declarations? Go for it, use them. Does it mean a pre-commit hook that rejects wonky indentation? Make it so. Does it mean using a programming language that uses variable declarations? Go for it. Or maybe it means using a language that DOESN'T use variable declarations. That's also an option. Ultimately, you use whatever helps you to produce the least-buggy code you can. No single feature is 100% necessary; no single feature is utterly useless. (And yes, the choices you make depend on context. Some tools are useful in large collaborations but pointless overhead in small projects, etc. Be smart, don't be rigid.) ChrisA From rosuav at gmail.com Sat Feb 17 00:31:19 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 16:31:19 +1100 Subject: Python 2 to 3 Conversion In-Reply-To: References: Message-ID: On Sat, Feb 17, 2018 at 4:15 PM, Wildman via Python-list wrote: > I have a bit of code I found on the web that will return > the ip address of the named network interface. The code > is for Python 2 and it runs fine. But, I want to use the > code with Python 3. Below is the code followed by the error > message. Suggestions appreciated. > > #!/usr/bin/env python3 > > import socket > import fcntl > import struct > > def get_ip_address(ifname): > s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) > return socket.inet_ntoa(fcntl.ioctl( > s.fileno(), > 0x8915, # SIOCGIFADDR > struct.pack('256s', ifname[:15]) > )[20:24]) > print(get_ip_address("eth0")) > print(get_ip_address("lo")) > > > Traceback (most recent call last): > File "./test.py", line 14, in > print(get_ip_address("eth0")) > File "./test.py", line 12, in get_ip_address > struct.pack('256s', ifname[:15]) > struct.error: argument for 's' must be a bytes object The easiest way is to encode it to ASCII: struct.pack('256s', ifname[:15].encode('ascii')) But you may want to see if there's a completely different way to do what you're looking at. I'm not sure what this is doing, so I can't advise more specifically. ChrisA From robertvstepp at gmail.com Sat Feb 17 01:25:03 2018 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 17 Feb 2018 00:25:03 -0600 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> Message-ID: I've just reread everyone's replies and one point you mentioned about the GIL caught my eye ... On Fri, Feb 16, 2018 at 11:16 PM, Chris Angelico wrote: > Asynchronicity and concurrency are hard. Getting your head around a > program that is simultaneously doing two things is inherently tricky. > Anything that simplifies this (such as providing atomic and > thread-safe operations on high-level data structures) will make this > sort of work FAR easier. That's what Python gives you. CPython's > current implementation (with the GIL) is reasonably easy to > conceptualize, and requires very few other locks to make everything > work sanely; all attempts to remove the GIL from CPython have involved > other tradeoffs, usually resulting in significant performance > penalties on single-threaded code. I am curious as to what efforts have been attempted to remove the GIL and what tradeoffs resulted and why? Is there a single article somewhere that collates this information? I fear if I try to Google this I will get a lot of scattered pieces that I will have to wade through to get to what I want to know, where you (or someone else) might be able to point me to a good link. Or kindly summarize yourself the relevant information. Thanks! -- boB From rosuav at gmail.com Sat Feb 17 01:54:52 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 17:54:52 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> Message-ID: On Sat, Feb 17, 2018 at 5:25 PM, boB Stepp wrote: > I've just reread everyone's replies and one point you mentioned about > the GIL caught my eye ... > > On Fri, Feb 16, 2018 at 11:16 PM, Chris Angelico wrote: > >> Asynchronicity and concurrency are hard. Getting your head around a >> program that is simultaneously doing two things is inherently tricky. >> Anything that simplifies this (such as providing atomic and >> thread-safe operations on high-level data structures) will make this >> sort of work FAR easier. That's what Python gives you. CPython's >> current implementation (with the GIL) is reasonably easy to >> conceptualize, and requires very few other locks to make everything >> work sanely; all attempts to remove the GIL from CPython have involved >> other tradeoffs, usually resulting in significant performance >> penalties on single-threaded code. > > I am curious as to what efforts have been attempted to remove the GIL > and what tradeoffs resulted and why? Is there a single article > somewhere that collates this information? I fear if I try to Google > this I will get a lot of scattered pieces that I will have to wade > through to get to what I want to know, where you (or someone else) > might be able to point me to a good link. Or kindly summarize > yourself the relevant information. > > Thanks! No, there isn't a single article, at least not that I know of. A good word to search for is "gilectomy", which brought me to this talk by Larry Hastings: https://www.youtube.com/watch?v=pLqv11ScGsQ Broadly speaking, what happens is that removing a large-scale lock (the GIL) requires using a whole lot of small-scale locks. That gives finer granularity, but it also adds a whole lot of overhead; the CPU features required for implementing those locks are not fast. With the GIL, you claim it, and you can do what you like. Without the GIL, you have to claim a lock on each object you manipulate, or something along those lines. (Different attempts have gone for different forms of granularity, so I can't generalize too much here.) That means claiming and relinquishing a lot more locks, which in turn means a lot more CPU-level "lock" primitives. That's a lot of overhead. One of the best ways to multi-thread CPU-intensive work is to push a lot of the work into an extension library. Take a function like this: def frobnosticate(stuff): magic magic magic magic magic more magic return result As long as the returned object is a newly-created one and the parameter is not mutated in any way, you can do all the work in the middle without holding the GIL. That can't be done in pure Python, but in a C function, it certainly can. You still have the coarse-grained locking of the GIL, you still have all the protection, but you can now have two threads frobnosticating different stuff at the same time. ChrisA From ben+python at benfinney.id.au Sat Feb 17 02:11:00 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 17 Feb 2018 18:11:00 +1100 Subject: Python 2 to 3 Conversion References: Message-ID: <85tvugay63.fsf@benfinney.id.au> Wildman via Python-list writes: > def get_ip_address(ifname): > s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) > return socket.inet_ntoa(fcntl.ioctl( > s.fileno(), > 0x8915, # SIOCGIFADDR > struct.pack('256s', ifname[:15]) > )[20:24]) > print(get_ip_address("eth0")) > print(get_ip_address("lo")) In Python 3, the literal ?"eth0"? creates a text (?unicode?) object. Python 3 correctly implements text as Unicode, which has no particular representation as bytes. You then pass that object to ?struct.pack('256s', ?)?, which (as the error says) expects a ?bytes? object. A text object is incompatible with that expectation. If you want a sequence of bytes, perhaps you want to encode the text, to some specified encoding. Maybe ASCII: struct.pack('256s', ifname.encode('ascii')) That may be different from what you want the code to do, though. It's not clear from the code what its intention is. -- \ ?What is needed is not the will to believe but the will to find | `\ out, which is the exact opposite.? ?Bertrand Russell, _Free | _o__) Thought and Official Propaganda_, 1928 | Ben Finney From robertvstepp at gmail.com Sat Feb 17 02:12:06 2018 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 17 Feb 2018 01:12:06 -0600 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> Message-ID: On Sat, Feb 17, 2018 at 12:54 AM, Chris Angelico wrote: > On Sat, Feb 17, 2018 at 5:25 PM, boB Stepp wrote: >> >> I am curious as to what efforts have been attempted to remove the GIL >> and what tradeoffs resulted and why? Is there a single article >> somewhere that collates this information? I fear if I try to Google >> this I will get a lot of scattered pieces that I will have to wade >> through to get to what I want to know, where you (or someone else) >> might be able to point me to a good link. Or kindly summarize >> yourself the relevant information. >> >> Thanks! > > No, there isn't a single article, at least not that I know of. A good > word to search for is "gilectomy", which brought me to this talk by > Larry Hastings: > > https://www.youtube.com/watch?v=pLqv11ScGsQ Thanks for the link. I'll give it a look tomorrow. Must get some sleep soon! Meanwhile I just finished reading "Efficiently Exploiting Multiple Cores with Python" (from Nick Coghlan's Python Notes) at http://python-notes.curiousefficiency.org/en/latest/python3/multicore_python.html It answered some of my questions and has a lot of good information. It appears that a "gilectomy" as you put it is a really tough problem. It looks to me that it might require a major backwards-compatibility-breaking change to CPython to implement if I am even close to understanding the issues involved. > Broadly speaking, what happens is that removing a large-scale lock > (the GIL) requires using a whole lot of small-scale locks. That gives > finer granularity, but it also adds a whole lot of overhead; the CPU > features required for implementing those locks are not fast. With the > GIL, you claim it, and you can do what you like. Without the GIL, you > have to claim a lock on each object you manipulate, or something along > those lines. (Different attempts have gone for different forms of > granularity, so I can't generalize too much here.) That means claiming > and relinquishing a lot more locks, which in turn means a lot more > CPU-level "lock" primitives. That's a lot of overhead. Thanks for this explanation. It really helps me a lot! > One of the best ways to multi-thread CPU-intensive work is to push a > lot of the work into an extension library. Take a function like this: > > def frobnosticate(stuff): > magic magic magic > magic magic more magic > return result > > As long as the returned object is a newly-created one and the > parameter is not mutated in any way, you can do all the work in the > middle without holding the GIL. That can't be done in pure Python, but > in a C function, it certainly can. You still have the coarse-grained > locking of the GIL, you still have all the protection, but you can now > have two threads frobnosticating different stuff at the same time. Yeah, Coghlan's article mentioned this. I guess using Cython would be one approach to this. Thanks, Chris! -- boB From auriocus at gmx.de Sat Feb 17 02:29:23 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 17 Feb 2018 08:29:23 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: Am 17.02.18 um 06:12 schrieb Stefan Ram: > Chris Angelico quotes: >>> Python is relatively slow compared to programming languages that >>> run closer to the operating system. The run time of the countdown >>> example above is orders of magnitude faster when implemented >>> in other language runtimes. > > What seems to be especially slow is printing many long > lines (from a loop) into the console of IDLE. > The guy in the article has not printed anything, but you are right, printing in IDLE is slow for lonng lines, especially. The reason is, that the Tk text widget is slow if the lines are long due to inferior layout algorithms. Gregor Cramer wrote a new implementation last year: https://core.tcl.tk/tips/doc/trunk/tip/466.md It's fully backwards compatible, orders of magnitude faster in specific circumstances and will appear in Tk 8.7. Testers are most welcome. Sadly, the Tcl/Tk developer community is extremely small. The new text widget was a big fuss last year, but I haven't heard anything about it since almost a year :( Christian From auriocus at gmx.de Sat Feb 17 02:50:38 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 17 Feb 2018 08:50:38 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> Message-ID: Am 17.02.18 um 05:54 schrieb boB Stepp: > And this is one I > am still puzzling over: Are statically-typed languages inherently > "safer" than properly implemented dynamically-typed languages? I can > see the advantages of catching type errors at compile time versus run > time. If you haven't tried so far, try to write a few small programs in Haskell. You will find that the excellent type system catches many errors at compile time. Haskell works very well for computations, like graph algorithms and such. My experience playing with it was that after getting the program to compile, most errors were usually eliminated. What remains are things like off-by-one errors, but also these are usually less if you write your algorithms recursively. OTOH Haskell is extremely tedious to use for anything I/O related, because inherently I/O is not "functional" and some extremely convoluted thing ("IO monads") had to be erected to make the nicely mathematical "pure" world interface with the dirty reality out there. Christian From tjreedy at udel.edu Sat Feb 17 03:29:49 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 17 Feb 2018 03:29:49 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 2/16/2018 10:22 PM, boB Stepp wrote: > This article is written by Nathan Murthy, a staff software engineer at > Tesla. The article is found at: > https://medium.com/@natemurthy/all-the-things-i-hate-about-python-5c5ff5fda95e To add to what other have said: Here is what the author said about word choice: " The word ?dynamically typed? has a positive connotation, more so than ?weakly typed.? Conversely, the word ?strongly typed? sounds more positive than saying ?statically typed.? Diction matters, because proponents of different camps will select a word-choice that reflects their bias of one programming language over another." Exactly. His dreadful strawperson code snippet should not be allowed even in a beginning programming class, let alone in professional programs. def foo(x): if is_valid(x): return "hello world" else: return bar(x) PEP 8 recommends meaningful names and a docstring that begins with "Return a ...". The meaningless names and lack of doc are what makes foo hard to read, not static versus dynamic typing. If you want to run on multiple cores, use multiple processes, either with multiprocessing or subprocess. Module multiprocessing intentionally has an api similar to threading, so one can convert at least some multiple-thread programs to multiple processes and cores. If you want safe programs, do not use unsafe weakly and statically typed C, which allows buffer overruns if one on uses the unsafe strxxx functions instead of the safer strnxxx functions. Buffer overruns may still be the most common exploit of viruses and other malware. Python coredevs try to be security conscious by adding security features and recommending good practices. We just cannot force people to upgrade and follow best practice. -- Terry Jan Reedy From martin.schoon at gmail.com Sat Feb 17 04:24:32 2018 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 17 Feb 2018 09:24:32 GMT Subject: Mayavi website? Message-ID: Anyone else having problems with interacting with http://code.enthought.com/pages/mayavi-project.html ? All I can see is the front page. Clicking on anything in that page results in nothing or 404. I have tried several browsers. /Martin From greg.ewing at canterbury.ac.nz Sat Feb 17 04:29:38 2018 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 17 Feb 2018 22:29:38 +1300 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: boB Stepp wrote: > "Python is viewed as a ubiquitous programming language; however, its > design limits its potential as a reliable and high performance systems > language. Unfortunately, not every developer is aware of its > limitations." "The Toyota Corolla is viewed as a ubiquitous family car; however, its design limits its potential for Formula 1 racing applications. Unfortunately, not every driver is aware of its limitations." Makes about as much sense. -- Greg From Cecil at decebal.nl Sat Feb 17 04:45:38 2018 From: Cecil at decebal.nl (Cecil Westerhof) Date: Sat, 17 Feb 2018 10:45:38 +0100 Subject: Replying on a tweet with Twython Message-ID: <87po546jb1.fsf@munus.decebal.nl> I just found Twython. I managed to post a tweet with: from twython import Twython twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET) posted = twitter.update_status(status = quote) But I want to send a reply on this tweet. I tried: posted = twitter.update_status(status = follow_up, in_reply_id = posted['id_str']) But that does not work. How could I make it work? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From marko at pacujo.net Sat Feb 17 05:52:35 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 17 Feb 2018 12:52:35 +0200 Subject: Are the critiques in "All the things I hate about Python" valid? References: Message-ID: <877erbsxak.fsf@elektro.pacujo.net> Gregory Ewing : > boB Stepp wrote: >> "Python is viewed as a ubiquitous programming language; however, its >> design limits its potential as a reliable and high performance >> systems language. Unfortunately, not every developer is aware of its >> limitations." > > "The Toyota Corolla is viewed as a ubiquitous family car; however, its > design limits its potential for Formula 1 racing applications. > Unfortunately, not every driver is aware of its limitations." > > Makes about as much sense. Python is near the sweet spot in expressivity. That means Python is great at complex, correct, maintainable, robust software. The primary downside is bad performance so at times you need to use some other programing language at the expense of robustness and productivity. Many people think static typing is key to high quality. I tend to think the reverse is true: the boilerplate of static typing hampers expressivity so much that, on the net, quality suffers. That said, Python doesn't prevent you from creating bad, ugly code. Also, not everything in Python and its ecosystem is solid gold. Python needs to be coupled with skilled hands and a good taste. Marko From ben.usenet at bsb.me.uk Sat Feb 17 06:28:08 2018 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sat, 17 Feb 2018 11:28:08 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? References: <877erbsxak.fsf@elektro.pacujo.net> Message-ID: <87mv07zwhj.fsf@bsb.me.uk> Marko Rauhamaa writes: > Many people think static typing is key to high quality. I tend to think > the reverse is true: the boilerplate of static typing hampers > expressivity so much that, on the net, quality suffers. I don't find that with Haskell. It's statically typed but the types are almost always inferred. If you see an explicit type, it's usually because the author thinks it helps explain something. (I don't want to start a Haskell/Python thread -- the only point is that static typing does not inevitably imply lots of 'boilerplate'.) -- Ben. From Cecil at decebal.nl Sat Feb 17 06:53:08 2018 From: Cecil at decebal.nl (Cecil Westerhof) Date: Sat, 17 Feb 2018 12:53:08 +0100 Subject: Replying on a tweet with Twython References: <87po546jb1.fsf@munus.decebal.nl> Message-ID: <87lgfr7ryz.fsf@munus.decebal.nl> Cecil Westerhof writes: > I just found Twython. I managed to post a tweet with: > from twython import Twython > twitter = Twython(APP_KEY, APP_SECRET, > OAUTH_TOKEN, OAUTH_TOKEN_SECRET) > posted = twitter.update_status(status = quote) > > But I want to send a reply on this tweet. I tried: > posted = twitter.update_status(status = follow_up, in_reply_id = posted['id_str']) > > But that does not work. How could I make it work? I need to use the following: posted = twitter.update_status(status = follow_up, in_reply_to_status_id = posted['id']) -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From rosuav at gmail.com Sat Feb 17 07:58:32 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Feb 2018 23:58:32 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <87mv07zwhj.fsf@bsb.me.uk> References: <877erbsxak.fsf@elektro.pacujo.net> <87mv07zwhj.fsf@bsb.me.uk> Message-ID: On Sat, Feb 17, 2018 at 10:28 PM, Ben Bacarisse wrote: > Marko Rauhamaa writes: > >> Many people think static typing is key to high quality. I tend to think >> the reverse is true: the boilerplate of static typing hampers >> expressivity so much that, on the net, quality suffers. > > I don't find that with Haskell. It's statically typed but the types are > almost always inferred. If you see an explicit type, it's usually > because the author thinks it helps explain something. > > (I don't want to start a Haskell/Python thread -- the only point is that > static typing does not inevitably imply lots of 'boilerplate'.) > Which goes to show just how misunderstood terms like "static typing" are. I wonder, is "static vs dynamic typing" on par with "call by reference vs call by value" in the way people pigeon-hole every programming language, not understanding that it's not a dichotomy? ChrisA From python.list at tim.thechases.com Sat Feb 17 08:47:18 2018 From: python.list at tim.thechases.com (Tim Chase) Date: Sat, 17 Feb 2018 07:47:18 -0600 Subject: Fw: [issue22167] iglob() has misleading documentation (does indeed store names internally) Message-ID: <20180217074718.0b9dfd05@bigbox.christie.dr> Has anybody else been getting unexpected/unsolicited emails from the Python bug-tracker? I'm not associated with (didn't submit/lurk/follow/sign-up-for) this bug or its notifications but somehow I'm getting messages on this particular issue. I've now received two notifications (both on this same bug) coming to my custom Python Bugs email address. When I check the issue's page, I'm not on the Nosy list so I don't really have a good way to unfollow because it doesn't think I'm following it. I *am* in the Python Bugs DB associated with other issues, I'm just confused how I ended up attached to this bug. Anybody else either receiving unsolicited bug notifications or happen to know how/why I'm getting these? Thanks, -tkc Begin forwarded message: Date: Sat, 17 Feb 2018 08:09:23 +0000 From: Serhiy Storchaka To: python.bugs@[redacted] Subject: [issue22167] iglob() has misleading documentation (does indeed store names internally) Serhiy Storchaka added the comment: Unfortunately issue25596 didn't change anything about this issue. iglob() still stores names (actually DirEntry objects) of all files in a directory before starting yielding the first of them. Otherwise we cold exceed the limit of open file descriptors. ---------- nosy: +serhiy.storchaka _______________________________________ Python tracker _______________________________________ From steve+comp.lang.python at pearwood.info Sat Feb 17 09:13:50 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 17 Feb 2018 14:13:50 +0000 (UTC) Subject: Replying on a tweet with Twython References: <87po546jb1.fsf@munus.decebal.nl> <87lgfr7ryz.fsf@munus.decebal.nl> Message-ID: On Sat, 17 Feb 2018 12:53:08 +0100, Cecil Westerhof wrote: > Cecil Westerhof writes: > >> I just found Twython. I managed to post a tweet with: >> from twython import Twython >> twitter = Twython(APP_KEY, APP_SECRET, >> OAUTH_TOKEN, OAUTH_TOKEN_SECRET) >> posted = twitter.update_status(status = quote) >> >> But I want to send a reply on this tweet. I tried: >> posted = twitter.update_status(status = follow_up, in_reply_id >> = posted['id_str']) >> >> But that does not work. How could I make it work? Define "does not work". What happens when you try? - the computer catches fire; - Blue Screen Of Death; - you get a Python traceback; - the police raid your house; - the computer says "I'm sorry Dave, I'm afraid I can't do that"; - something else? If you get a traceback, don't keep it a secret, please copy and paste it so we can read it and possibly help. If you *don't* get a traceback, what value does `posted` get? Please run this: print(posted, type(posted)) > I need to use the following: > posted = twitter.update_status(status = follow_up, > in_reply_to_status_id = posted['id']) Okay. What happens when you try? -- Steve From steve+comp.lang.python at pearwood.info Sat Feb 17 09:30:24 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 17 Feb 2018 14:30:24 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: Message-ID: On Sat, 17 Feb 2018 03:29:49 -0500, Terry Reedy wrote: > His dreadful strawperson code snippet should not be allowed even in a > beginning programming class, let alone in professional programs. > > def foo(x): > if is_valid(x): > return "hello world" > else: > return bar(x) Indeed. Well said Terry. The author complains that we don't know what function foo() does, as if that's a problem with the Python language. Of course we can write crap code in any language: the function has a ludicrous name, a badly named argument, and no documentation or comments. But he pretends that this is a limitation of Python. Not cool. What's this Pascal function do? (I'm more familiar with Pascal than C, but I'm sure some kind person can translate it into the equivalent idiomatic C.) function foo(x: integer): Str256; begin if is_valid(x): foo := 'hello world'; else: foo := bar(x); end; Apart from now knowing that x is an integer, and that it returns a string, we're just as much in the dark, despite the static type system. What's the meaning of the function? What counts as valid input and output? No idea. And the static type checker won't tell you. It won't complain that the function is called "foo" or that the argument is "x" or that is_valid actually returns the result of a random flip-toss but only on Tuesdays, the rest of the time it always returns True. And ironically, in a *modern* statically typed language, you may not even need the type declarations. After all, in a modern type-checker, the compiler can infer that since foo returns 'hello world', it must return a string; it can probably even infer the type of the input. So in such a language, the source code may not even display the input or output types, because the compiler can infer what they logically must be. The only difference between such a modern statically typed language with no (or very few) type declarations, and Python, is that Python checks the types at run-time and the statically typed language checks them at compile-time. That gives a pros and cons: Pros for static typing: - catch some errors sooner; - allow the compiler to make some optimizations resulting in faster code; - a sufficiently sound type system may be able to prove that a certain class of bugs (type bugs) cannot occur; (assuming you trust that the compiler is bug-free). Cons for static typing: - programmer may have to include lots of tedious code to placate the compiler; - programmer may have to fight the compiler to force it to allow perfectly safe operations which the type-checker disallows; - type inference is not always fool-proof; - typically badly suited to interactive or exploratory programming, and less flexible; - may give the programmer a false sense of security that just because the program compiles, it must be bug-free. There's a reason we often joke about programmers saying "It compiles, quick, let's ship it!". -- Steve From ian.g.kelly at gmail.com Sat Feb 17 09:47:12 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 17 Feb 2018 07:47:12 -0700 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Fri, Feb 16, 2018 at 9:32 PM, Chris Angelico wrote: > You'd be surprised how rarely that kind of performance even matters. > The author of that article cites C# as a superior language, but in the > rewrite from C# to Python (the same one I mentioned in the other > post), I sped the program up incredibly. Part of that is because C# > requires the startup of its framework (in my case that's Mono) just as > Python does, and partly it's because the simplicity of Python let me > eliminate a number of unnecessary HTTP requests. Trust me, your choice > of language doesn't help you if it means you do three (sequential) > HTTP requests when one would have done. Clean code has its own > advantages. Okay, I'm curious. How did C# force you to make extra HTTP requests that were no longer necessary when you rewrote in Python? From steve+comp.lang.python at pearwood.info Sat Feb 17 09:47:30 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 17 Feb 2018 14:47:30 +0000 (UTC) Subject: How to run script from interpreter? References: Message-ID: On Fri, 16 Feb 2018 14:18:16 -0800, windhorn wrote about running scripts from the interpreter: [...] > That works, but nothing is very convenient for debugging simple scripts. > If I run the script from a command prompt it works, but I lose all my > other stuff (debugging functions, variables, etc.). I'm sorry, reading your description of the various steps you tried brought to my mind a chef insisting that the only cooking implement he needs is a spoon, and then complaining that it's not well suited to slicing meat :-) The interactive interpreter, or REPL (Read-Eval-Print Loop) is not designed for the edit-run-debug-edit cycle. But it can be used as a single component of that development process. For me, the tool I use is a set of re-usable tools: - a text editor; - a system command prompt in a terminal/console window; - a Python REPL for running code snippets and looking up help(obj). Other people prefer a IDE like Spyder (or many others). Either way, they're designed for the edit-run-debug-edit cycle. I edit the source code of my script in the text editor, then run it from the command prompt using python myscript.py If I need the debugger, I use: python -i myscript.py I often use test driven development, so I have a set of tests written. I edit the tests, then run the test suite: python myscript_tests.py then edit the script to make the tests work. The best part of this is each time you run the script, it is guaranteed to be running *fresh*, with no left-over cruft from previous runs. Trust me, after you've spent hours trying to debug a mysterious problem in your code, only to discover that it was a side-effect of re-using the same interactive session that ran an older version of your code, you too will shudder at the thought of trying to do all your development in an on- going, persistent REPL session. > More a comment than a question but seems like sometimes execfile() is > the right tool. I'm sure that it is; but the question, the right tool for what? -- Steve From bc at freeuk.com Sat Feb 17 10:36:22 2018 From: bc at freeuk.com (bartc) Date: Sat, 17 Feb 2018 15:36:22 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 17/02/2018 14:30, Steven D'Aprano wrote: > And ironically, in a *modern* statically typed language, you may not even > need the type declarations. After all, in a modern type-checker, the > compiler can infer that since foo returns 'hello world', it must return a > string; it can probably even infer the type of the input. At the call-site, it would need to see the function definition, or some declaration (with static types) that the compiler has generated from the definition. That implies a module system that I think has priority over type inference. For many uses, type declarations will still be needed: for function calls across languages, or to create a particular layout of structure or specialist data structure. What can /very/ easily be done, even in Python, is to have two kinds of function: one an actual function that returns a value, and the other a procedure that returns no value. Then it is easy to pick up errors where no return value is used, or a return value is supplied when it isn't needed. (In Python, procedures will always have to return None behind the scenes, as code might still call procedures as functions.) So in such a > language, the source code may not even display the input or output types, > because the compiler can infer what they logically must be. > > The only difference between such a modern statically typed language with > no (or very few) type declarations, and Python, is that Python checks the > types at run-time and the statically typed language checks them at > compile-time. That gives a pros and cons: > > Pros for static typing: > > - catch some errors sooner; > > - allow the compiler to make some optimizations resulting in > faster code; > > - a sufficiently sound type system may be able to prove that > a certain class of bugs (type bugs) cannot occur; I've done a lot of conversion from dynamic to static code. I found the extra discipline needed very useful. I could no longer pass multiple types to one function, or return a range of types, but the end result was much tidier code, where parameters and return types were better defined and the functions simpler. Needing to think about such things was better for avoiding sloppy, inefficient code. And the result was usually faster (even above the benefit expected of moving from compiled to interpreted). > > (assuming you trust that the compiler is bug-free). > > Cons for static typing: > > - programmer may have to include lots of tedious code to > placate the compiler; > > - programmer may have to fight the compiler to force it to > allow perfectly safe operations which the type-checker > disallows; > > - type inference is not always fool-proof; > > - typically badly suited to interactive or exploratory > programming, and less flexible; > > - may give the programmer a false sense of security that > just because the program compiles, it must be bug-free. If the static language doesn't have flexible arrays, that can be a bit of a bummer. -- bartc From breamoreboy at gmail.com Sat Feb 17 11:51:26 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Sat, 17 Feb 2018 16:51:26 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 17/02/18 09:29, Gregory Ewing wrote: > boB Stepp wrote: >> "Python is viewed as a ubiquitous programming language; however, its >> design limits its potential as a reliable and high performance systems >> language. Unfortunately, not every developer is aware of its >> limitations." > > "The Toyota Corolla is viewed as a ubiquitous family car; however, > its design limits its potential for Formula 1 racing applications. > Unfortunately, not every driver is aware of its limitations." > > Makes about as much sense. > +1 QOTW. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From best_lay at yahoo.com Sat Feb 17 12:35:32 2018 From: best_lay at yahoo.com (Wildman) Date: Sat, 17 Feb 2018 11:35:32 -0600 Subject: Python 2 to 3 Conversion References: Message-ID: Thanks to Chris and Ben. Your suggestions were slightly different but both worked equally well, although I don't understand how that can be so. > struct.pack('256s', ifname[:15].encode('ascii')) > struct.pack('256s', ifname.encode('ascii')) I was looking for a reliable way to determine the IP addy for a given network adapter. That is what the code does but I don't understand how it does it either. Thanks again. I will continue to research and study the code in the hope I will understand it. -- GNU/Linux user #557453 The cow died so I don't need your bull! From steve+comp.lang.python at pearwood.info Sat Feb 17 13:05:27 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 17 Feb 2018 18:05:27 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: Message-ID: On Sat, 17 Feb 2018 15:25:15 +1100, Chris Angelico wrote: > 1) Type safety. > > This is often touted as a necessity for industrial-grade software. It > isn't. There are many things that a type system, no matter how > sophisticated, cannot catch; The usual response to that is to make ever-finer-grained types, until the type-system can prove the code is correct. integers positive integers positive integers greater than 10 positive integers greater than 10 but less than 15003 positive odd integers greater than 10 but less than 15003 positive odd integers greater than 10 but less than 15003 divisible by 17 Of course, this has a few minor (ha!) difficulties... starting with the hardest problem in computer science, naming things. Even if you can come up with unique, concise names for these types that won't overwhelm the reader, it isn't clear that the type system will always be capable of representing such fine distinctions. How would you specify two string types, one for personal names and one for family names, so that the compiler can detect any attempt to assign a family name to a personal name, or vise versa? And of course, there are certain errors which *even in principle* cannot be detected at compile-time: - bad user input; - memory exhaustion; - out of disk space; - I/O errors when reading or writing data; - network failures; - logic errors (your code does what told it to do perfectly, it's just not what you wanted...) etc. > for some reason, though, we don't hear > people saying "C is useless for industrial-grade software because it > doesn't have function contracts". You obviously don't speak to Eiffel programmers then :-) > Anyway, if you want some system of type checking, you can use static > analysis (eg tools like MyPy) to go over your code the same way a > compiler might. Indeed. Despite our criticisms of the *attitude* that static typing is a panacea, it must be recognised that it is useful, and the bigger the project, the more useful it is. And some type checkers are *very* impressive. Google for "compiler found my infinite loop" for a classic example of a compiler detecting at compile-time than a while loop would never terminate. (The language was ML, not C. The C type system isn't that good. In fact, ironically proponents of languages in the ML family consider C and its family to be weakly-typed.) I can understand people saying that for sufficiently large projects, they consider it indispensable to have the assistance of a type checker. That in and of itself is no worse than saying that, as a writer, I find a spell checker to be indispensable. > "The first glaring issue is that I have no guarantee that is_valid() > returns a bool type." -- huh? It's being used in a boolean context, and > it has a name that starts "is_". How much guarantee are you looking for? > *ANY* object can be used in an 'if', so it doesn't even matter. This is > a stupidly contrived criticism. I don't think so -- I think a lot of people really have difficulty coming to terms with Python's duck-typing of bools. They just don't like, or possibly even grok, the idea of truthy and falsey values, and want the comfort of knowing that the value "really is" a True or False. We can come up with some contrived justifications for this... what if is_valid() contains a bug: def is_valid(arg): if some condition: return arg # oops I meant True return False then static analysis would detect this. With truthiness, you can't tell: what if *nearly* all the input args just happen to be truthy? Then the code will nearly always work, and the errors will be perplexing. But I consider that a fairly contrived scenario, and one with at least two alternate solutions: code review, and unit tests. But still, I do see the point that a static analyser could have picked up that error even if you didn't have code review, even if the person writing the unit tests never imagined this failure mode. [...] > Totally not true. The GIL does not stop other threads from running. > Also, Python has existed for multiple CPU systems pretty much since its > inception, I believe. (Summoning the D'Aprano for history lesson?) If you're talking about common desktop computers, I think you're forgetting how recent multicore machines actually are. I'm having difficulty finding when multicore machines first hit the market, but it seems to have been well into the 21st century -- perhaps as late as 2006 with the AMD Athelon 64 X2: https://www.computerhope.com/history/processor.htm The first public release of Python was 1991, so I'm not sure what sort of support it had for multiple CPUs or multiple cores. Perhaps none. By the way, multiple CPU machines are different from CPUs with multiple cores: http://smallbusiness.chron.com/multiple-cpu-vs-multicore-33195.html One of the more interesting (although perhaps not cost-effective) uses for multiple CPUs is a machine with two distinct hardware architectures, running two distinct OSes, well before the days of virtualisation technology. Back in the 1980s (1989, perhaps?) TI and Apple developed a NuBus card containing a TI Lisp Machine CPU. For a mere $10,000 or so ($20K in today's money) you could simultaneously run a Lisp Machine in your Macintosh II. http://lemonodor.com/archives/2002/10/ti_microexplore.html Certainly though there have been versions of Python without a GIL for a long time: Jython started as JPython, in 1997; IronPython was started around 2003 or so, and reached the 1.0 milestone in 2006. Fun fact: (then) Microsoft engineer Jim Hugunin created both JPython and IronPython! -- Steve From giuseppecostanzi at gmail.com Sat Feb 17 13:32:12 2018 From: giuseppecostanzi at gmail.com (Beppe) Date: Sat, 17 Feb 2018 10:32:12 -0800 (PST) Subject: [Tkinter]Validation to multiple Entry widgets Message-ID: hi all, I would validate values input, on key, in multiple Entry widgets create at run time I wrote this snip but even if the entry are created and the callback work well the relative value is missing #!/usr/bin/python3 import tkinter as tk class Application(tk.Frame): def __init__(self, master=None): super().__init__(master) self.vcmd = (self.register(self.validate), '%d', '%i', '%P', '%s', '%S', '%v', '%V', '%W') self.pack() self.init_ui() def init_ui(self): my_list = (2.14,18.3,76.4,2.38,0.425,2.68,1.09,382,8.59,0.495) for i in my_list: #this work #w = tk.Entry(self, bg='white') #this doesn't work w = tk.Entry(self, bg='white', validate = 'key', validatecommand = self.vcmd) w.insert(tk.END, i) w.pack() def validate(self, action, index, value_if_allowed, prior_value, text, validation_type, trigger_type, widget_name): # action=1 -> insert if(action=='1'): if text in '0123456789.-+': try: float(value_if_allowed) return True except ValueError: return False else: return False else: return True root = tk.Tk() app = Application(master=root) app.mainloop() tips? regards beppe From __peter__ at web.de Sat Feb 17 14:20:41 2018 From: __peter__ at web.de (Peter Otten) Date: Sat, 17 Feb 2018 20:20:41 +0100 Subject: [Tkinter]Validation to multiple Entry widgets References: Message-ID: Beppe wrote: > I would validate values input, on key, in multiple Entry widgets create at > run time > > I wrote this snip but even if the entry are created and the callback work > well the relative value is missing > my_list = (2.14,18.3,76.4,2.38,0.425,2.68,1.09,382,8.59,0.495) > > for i in my_list: > #this work > #w = tk.Entry(self, bg='white') > #this doesn't work > w = tk.Entry(self, bg='white', validate = 'key', > validatecommand = self.vcmd) w.insert(tk.END, i) > w.pack() > > def validate(self, action, index, value_if_allowed, > prior_value, text, validation_type, > trigger_type, widget_name): > # action=1 -> insert When you add print(text) here to debug your code the old-fashioned way you'll note that text is, e. g. '2.14' and thus will not pass the `text in ...` test below. > if(action=='1'): > if text in '0123456789.-+': > try: > float(value_if_allowed) > return True > except ValueError: > return False > else: > return False > else: > return True > > > root = tk.Tk() > app = Application(master=root) > app.mainloop() > > > tips? > > regards beppe From rosuav at gmail.com Sat Feb 17 15:11:59 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Feb 2018 07:11:59 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sun, Feb 18, 2018 at 1:47 AM, Ian Kelly wrote: > On Fri, Feb 16, 2018 at 9:32 PM, Chris Angelico wrote: >> You'd be surprised how rarely that kind of performance even matters. >> The author of that article cites C# as a superior language, but in the >> rewrite from C# to Python (the same one I mentioned in the other >> post), I sped the program up incredibly. Part of that is because C# >> requires the startup of its framework (in my case that's Mono) just as >> Python does, and partly it's because the simplicity of Python let me >> eliminate a number of unnecessary HTTP requests. Trust me, your choice >> of language doesn't help you if it means you do three (sequential) >> HTTP requests when one would have done. Clean code has its own >> advantages. > > Okay, I'm curious. How did C# force you to make extra HTTP requests > that were no longer necessary when you rewrote in Python? It didn't *force* those requests to be made, but the code was so large and convoluted that I doubt its original author realized that the requests were being repeated. Pseudo-coded: def do_stuff(): if need_to_prepare(): prepare() really_do_stuff() def need_to_prepare(): requests.get("...") return True or False based on the response def really_do_stuff(): requests.get("same as need_to_prepare") if response says we need to prepare: prepare(), but only the bits we need requests.post("...") This whole structure was split across a number of separate files, making it highly unobvious that the request inside really_do_stuff is actually redundant in this context. (Plus, I could actually skip the check altogether and just do the work. If we need to do the preparation, that can be overlaid with a second request, done in response to a failure. Max of two HTTP requests, and usually just one.) By making the code MASSIVELY simpler, Python allowed me (as the programmer) to see where improvements could be made. When you have thousands upon thousands of lines of code, it's far harder to recognize where one function's job overlaps another's, and harder still to figure out a clean way to reduce the duplication without breaking everything. ChrisA From giuseppecostanzi at gmail.com Sat Feb 17 15:14:38 2018 From: giuseppecostanzi at gmail.com (Beppe) Date: Sat, 17 Feb 2018 12:14:38 -0800 (PST) Subject: [Tkinter]Validation to multiple Entry widgets In-Reply-To: References: Message-ID: <6286e7fc-7a8b-49f3-b78a-7b47740da4de@googlegroups.com> Il giorno sabato 17 febbraio 2018 20:21:53 UTC+1, Peter Otten ha scritto: > Beppe wrote: > > > I would validate values input, on key, in multiple Entry widgets create at > > run time > > > > I wrote this snip but even if the entry are created and the callback work > > well the relative value is missing > > > my_list = (2.14,18.3,76.4,2.38,0.425,2.68,1.09,382,8.59,0.495) > > > > for i in my_list: > > #this work > > #w = tk.Entry(self, bg='white') > > #this doesn't work > > w = tk.Entry(self, bg='white', validate = 'key', > > validatecommand = self.vcmd) w.insert(tk.END, i) > > w.pack() > > > > def validate(self, action, index, value_if_allowed, > > prior_value, text, validation_type, > > trigger_type, widget_name): > > # action=1 -> insert > > When you add > > print(text) > > here to debug your code the old-fashioned way you'll note that text is, e. > g. '2.14' and thus will not pass the `text in ...` test below. yeah, thank you very much, I change if text in '0123456789.-+': with if text: and it works now thank you, Peter > > > if(action=='1'): > > if text in '0123456789.-+': > > try: > > float(value_if_allowed) > > return True > > except ValueError: > > return False > > else: > > return False > > else: > > return True > > > > > > root = tk.Tk() > > app = Application(master=root) > > app.mainloop() > > > > > > tips? > > > > regards beppe From Cecil at decebal.nl Sat Feb 17 15:27:52 2018 From: Cecil at decebal.nl (Cecil Westerhof) Date: Sat, 17 Feb 2018 21:27:52 +0100 Subject: Replying on a tweet with Twython References: <87po546jb1.fsf@munus.decebal.nl> <87lgfr7ryz.fsf@munus.decebal.nl> Message-ID: <87371z7453.fsf@munus.decebal.nl> Steven D'Aprano writes: > On Sat, 17 Feb 2018 12:53:08 +0100, Cecil Westerhof wrote: > >> Cecil Westerhof writes: >> >>> I just found Twython. I managed to post a tweet with: >>> from twython import Twython >>> twitter = Twython(APP_KEY, APP_SECRET, >>> OAUTH_TOKEN, OAUTH_TOKEN_SECRET) >>> posted = twitter.update_status(status = quote) >>> >>> But I want to send a reply on this tweet. I tried: >>> posted = twitter.update_status(status = follow_up, in_reply_id >>> = posted['id_str']) >>> >>> But that does not work. How could I make it work? > > Define "does not work". What happens when you try? It is posted as a standalone tweet instead of as a reply on the first tweet. >> I need to use the following: >> posted = twitter.update_status(status = follow_up, >> in_reply_to_status_id = posted['id']) > > Okay. What happens when you try? Then the tweet is posted as a reply on the the first tweet. (So problem solved.) -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From rosuav at gmail.com Sat Feb 17 16:02:10 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Feb 2018 08:02:10 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sun, Feb 18, 2018 at 5:05 AM, Steven D'Aprano wrote: > On Sat, 17 Feb 2018 15:25:15 +1100, Chris Angelico wrote: > >> 1) Type safety. >> >> This is often touted as a necessity for industrial-grade software. It >> isn't. There are many things that a type system, no matter how >> sophisticated, cannot catch; > > The usual response to that is to make ever-finer-grained types, until the > type-system can prove the code is correct. > > integers > positive integers > positive integers greater than 10 > positive integers greater than 10 but less than 15003 > positive odd integers greater than 10 but less than 15003 > positive odd integers greater than 10 but less than 15003 divisible by 17 > > Of course, this has a few minor (ha!) difficulties... starting with the > hardest problem in computer science, naming things. Naming things isn't a problem if we're working with a type inference system. On the flip side, if your last example is purely type inference, it's not really a type checking system - it's a holistic static analysis. You can't say "TypeError: spamminess must be less than 15003" without also saying "oh but that might be a bug in this function, since it's meant to be able to take numbers >= 15003". Some of the type names CAN be generated algebraically. For instance, Pike lets you declare that something is an "int", or "array(int)" (an array of integers), or "int(1..)" (integer, minimum of 1, no maximum - in other words, positive integer), or "array(int(1..))" (yup, array of positive integers). You could probably devise a system like "int(11..15002|1%2|0%17)" to mean "must be between 11 and 15002, and must equal 1, modulo 2, and must equal 0, modulo 17". I'm not sure how often it would be of value, though, and it's pretty ugly. > Even if you can come up with unique, concise names for these types that > won't overwhelm the reader, it isn't clear that the type system will > always be capable of representing such fine distinctions. How would you > specify two string types, one for personal names and one for family > names, so that the compiler can detect any attempt to assign a family > name to a personal name, or vise versa? That's where the type system breaks down and the variable naming system shines. My favourite example here is of collections (which should be named in the plural) and their elements (which usually won't be). For instance: for msg in msgs: for person in people: for character in disney_princesses: for item in recipe.ingredients: No type system can reliably figure out that "msg" is singular and "msgs" is plural. And the concrete data types might even be identical ("msgs" could be a dict mapping message IDs to their actual messages, and then "msg" could be a dict mapping headers to their values - "for msg in msgs.values():" would thus iterate through one dict, yielding other dicts), even though *to the programmer* they are completely different, so type inference would need a lot of help. >> for some reason, though, we don't hear >> people saying "C is useless for industrial-grade software because it >> doesn't have function contracts". > > You obviously don't speak to Eiffel programmers then :-) True, I don't, but I'm not surprised there are people who think that way. But how many people write blog posts like the one that sparked this thread, clickbaitingly describing C as useless for serious work? >> Anyway, if you want some system of type checking, you can use static >> analysis (eg tools like MyPy) to go over your code the same way a >> compiler might. > > Indeed. Despite our criticisms of the *attitude* that static typing is a > panacea, it must be recognised that it is useful, and the bigger the > project, the more useful it is. And some type checkers are *very* > impressive. Google for "compiler found my infinite loop" for a classic > example of a compiler detecting at compile-time than a while loop would > never terminate. Exactly. Though there's a bit of a blurring now between "type checking" and "holistic static analysis". I've seen some incredible discoveries by Coverity; extremely narrow situational bugs where, if this happens and that fails and thingy was exactly 47, then the response message might use one byte more space than its buffer. That's pretty useful and seriously impressive, but I'm terrified of any sort of "type system" that could actually give a NAME to the data type that shows up this bug. > I can understand people saying that for sufficiently large projects, they > consider it indispensable to have the assistance of a type checker. That > in and of itself is no worse than saying that, as a writer, I find a > spell checker to be indispensable. Hmm, I do think there are a lot of people who take lessons learned on gigantic projects with huge contributor teams, and then say "EVERY program needs these tools". When you write a 200-page book, you might find an automated table of contents to be, not simply a useful tool, but an absolute necessity. Great! But when you write a one-screen README, that TOC creator is useless, along with the boilerplate in your document to tell it what to do. (Can you imagine adding a type checker to bash scripting?) >> "The first glaring issue is that I have no guarantee that is_valid() >> returns a bool type." -- huh? It's being used in a boolean context, and >> it has a name that starts "is_". How much guarantee are you looking for? >> *ANY* object can be used in an 'if', so it doesn't even matter. This is >> a stupidly contrived criticism. > > I don't think so -- I think a lot of people really have difficulty coming > to terms with Python's duck-typing of bools. They just don't like, or > possibly even grok, the idea of truthy and falsey values, and want the > comfort of knowing that the value "really is" a True or False. Okay, but even if you don't grok the truthy/falsey concept, it says "is_valid". Unless you're expecting outright MALICIOUS code, you should be able to assume that "is_valid" returns a boolean. Oh wait. We're talking about programmers here. Malicious has nothing on the rampant stupidity... https://thedailywtf.com/articles/What_Is_Truth_0x3f_ Still, it's not a fault of *this* function if it expects the normal case. It's no worse to expect is_valid to return a boolean (or something usable in a boolean context) than to expect math.sqrt to return a non-negative number. Sure, sqrt might have a bug in it so it returns a negative... but that's not your problem. > We can come up with some contrived justifications for this... what if > is_valid() contains a bug: > > def is_valid(arg): > if some condition: > return arg # oops I meant True > return False > > then static analysis would detect this. With truthiness, you can't tell: > what if *nearly* all the input args just happen to be truthy? Then the > code will nearly always work, and the errors will be perplexing. Right, and that's either a bug, or a design flaw (maybe the returning of 'arg' was intentional, because the author thought that ALL these args were truthy). That's fine. The name implies that it's returning a boolean, so you can look at this function *on its own* and pinpoint the (potential) problem. > But I consider that a fairly contrived scenario, and one with at least > two alternate solutions: code review, and unit tests. Exactly. Also, this sort of thing DOES happen, so if ever you add a __bool__ method to an object, consider the implications. > But still, I do see the point that a static analyser could have picked up > that error even if you didn't have code review, even if the person > writing the unit tests never imagined this failure mode. Perhaps. On the flip side, unless you rigidly demand that "if" statements must ONLY operate on the two values True and False, you still won't detect these problems at the calling site. You might detect them inside is_valid (if you declare that it'll return bool, and then return something that's not a bool, poof, error), but in the original complaint, type checking can't help without straitjacketing conditionals. >> Totally not true. The GIL does not stop other threads from running. >> Also, Python has existed for multiple CPU systems pretty much since its >> inception, I believe. (Summoning the D'Aprano for history lesson?) > > If you're talking about common desktop computers, I think you're > forgetting how recent multicore machines actually are. I'm having > difficulty finding when multicore machines first hit the market, but it > seems to have been well into the 21st century -- perhaps as late as 2006 > with the AMD Athelon 64 X2: No, I'm talking about big iron. Has Python been running on multi-CPU supercomputers earlier than that? > By the way, multiple CPU machines are different from CPUs with multiple > cores: > > http://smallbusiness.chron.com/multiple-cpu-vs-multicore-33195.html Yeah, it was always "multiple CPUs", not "multiple cores" when I was growing up. And it was only ever in reference to the expensive hardware that I could never even dream of working with. I was always on the single-CPU home-grade systems. > Certainly though there have been versions of Python without a GIL for a > long time: > > Jython started as JPython, in 1997; IronPython was started around 2003 or > so, and reached the 1.0 milestone in 2006. > > Fun fact: (then) Microsoft engineer Jim Hugunin created both JPython and > IronPython! Yes. I'm not sure how Jython handles concurrency; I'm totally in the dark about IronPython. I suspect both of them let the underlying system handle it, but that doesn't help me as I don't know that either. How well do they handle the "two threads spinning, incrementing the same global" stress test? I doubt they'll improve the efficiency. ChrisA From bc at freeuk.com Sat Feb 17 16:50:19 2018 From: bc at freeuk.com (bartc) Date: Sat, 17 Feb 2018 21:50:19 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 17/02/2018 20:11, Chris Angelico wrote: > On Sun, Feb 18, 2018 at 1:47 AM, Ian Kelly wrote: >> Okay, I'm curious. How did C# force you to make extra HTTP requests >> that were no longer necessary when you rewrote in Python? > > It didn't *force* those requests to be made, but the code was so large > and convoluted that I doubt its original author realized that the > requests were being repeated. .... > By making the code MASSIVELY simpler, Python allowed me (as the > programmer) to see where improvements could be made. When you have > thousands upon thousands of lines of code, it's far harder to > recognize where one function's job overlaps another's, and harder > still to figure out a clean way to reduce the duplication without > breaking everything. That's a very interesting observation. I've frequently made the complaint about systems that I consider large and complex also leading to such issues, where no one individual can see the whole picture. For example, in the system used for building CPython from source. But I guess what you're describing doesn't apply in such cases. Those 20K or 30K lines of configure scripts really /are/ necessary! -- bartc From rosuav at gmail.com Sat Feb 17 17:09:55 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Feb 2018 09:09:55 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sun, Feb 18, 2018 at 8:50 AM, bartc wrote: > On 17/02/2018 20:11, Chris Angelico wrote: >> >> On Sun, Feb 18, 2018 at 1:47 AM, Ian Kelly wrote: > > >>> Okay, I'm curious. How did C# force you to make extra HTTP requests >>> that were no longer necessary when you rewrote in Python? >> >> >> It didn't *force* those requests to be made, but the code was so large >> and convoluted that I doubt its original author realized that the >> requests were being repeated. > > > .... > >> By making the code MASSIVELY simpler, Python allowed me (as the >> programmer) to see where improvements could be made. When you have >> thousands upon thousands of lines of code, it's far harder to >> recognize where one function's job overlaps another's, and harder >> still to figure out a clean way to reduce the duplication without >> breaking everything. > > > That's a very interesting observation. > > I've frequently made the complaint about systems that I consider large and > complex also leading to such issues, where no one individual can see the > whole picture. > > For example, in the system used for building CPython from source. > > But I guess what you're describing doesn't apply in such cases. Those 20K or > 30K lines of configure scripts really /are/ necessary! > No, it does apply. But those 20-30K lines aren't what you should be reading. Here's the list of top-level files in the cpython repository, with their line counts: rosuav at sikorsky:~/cpython$ wc -l `git ls-files|grep -v /` 39 .gitattributes 115 .gitignore 58 .hgeol 106 .hgignore 182 .hgtags 146 .travis.yml 254 LICENSE 1772 Makefile.pre.in 253 README.rst 291 aclocal.m4 1473 config.guess 1836 config.sub 18321 configure 5576 configure.ac 294 install-sh 1583 pyconfig.h.in 2361 setup.py 34660 total The largest one here is "configure", but that's only in the repository to remove a dependency. You can delete it and then run "autoconf" and it'll be regenerated from configure.ac. With that removed, the largest file is about 5K lines, and all these files put together make up only 16K lines. If you're reading 30,000 lines of configuration script, you're basically reading object code. It's like calling dis.dis() on a Python function and then complaining that it's long and unreadable. Stop making unfair criticisms if you want people to take you seriously. ChrisA From bc at freeuk.com Sat Feb 17 19:13:41 2018 From: bc at freeuk.com (bartc) Date: Sun, 18 Feb 2018 00:13:41 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 17/02/2018 22:09, Chris Angelico wrote: > On Sun, Feb 18, 2018 at 8:50 AM, bartc wrote: >> That's a very interesting observation. >> >> I've frequently made the complaint about systems that I consider large and >> complex also leading to such issues, where no one individual can see the >> whole picture. >> >> For example, in the system used for building CPython from source. >> >> But I guess what you're describing doesn't apply in such cases. Those 20K or >> 30K lines of configure scripts really /are/ necessary! >> > 18321 configure > The largest one here is "configure", but that's only in the repository > to remove a dependency. You can delete it and then run "autoconf" and > it'll be regenerated from configure.ac. With that removed, the largest > file is about 5K lines, and all these files put together make up only > 16K lines. If you're reading 30,000 lines of configuration script, > you're basically reading object code. Try looking inside it. It's not object code. It's a bash script that seems to doing a lot of pointless stuff. And why is it necessary if it's generated from the configure.ac; they could just supply that. Not that that seems to be doing anything less doing any less pointless. It's like calling dis.dis() on a > Python function and then complaining that it's long and unreadable. > > Stop making unfair criticisms if you want people to take you seriously. My point about systems so complex that you can't understand the whole thing still stands, and it applies to a lot of things I come across. The CPython source is just a topical example. Do /you/ know for sure that there is no dead weight in there? Or do you just accept it because, after all, the thing works [for you], and it doesn't matter if it's bigger and slower than it needs to be. -- bartc From rosuav at gmail.com Sat Feb 17 19:45:35 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Feb 2018 11:45:35 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sun, Feb 18, 2018 at 11:13 AM, bartc wrote: > On 17/02/2018 22:09, Chris Angelico wrote: >> >> On Sun, Feb 18, 2018 at 8:50 AM, bartc wrote: > > >>> That's a very interesting observation. >>> >>> I've frequently made the complaint about systems that I consider large >>> and >>> complex also leading to such issues, where no one individual can see the >>> whole picture. >>> >>> For example, in the system used for building CPython from source. >>> >>> But I guess what you're describing doesn't apply in such cases. Those 20K >>> or >>> 30K lines of configure scripts really /are/ necessary! >>> > >> 18321 configure > > >> The largest one here is "configure", but that's only in the repository >> to remove a dependency. You can delete it and then run "autoconf" and >> it'll be regenerated from configure.ac. With that removed, the largest >> file is about 5K lines, and all these files put together make up only >> 16K lines. If you're reading 30,000 lines of configuration script, >> you're basically reading object code. > > > Try looking inside it. It's not object code. It's a bash script that seems > to doing a lot of pointless stuff. And why is it necessary if it's generated > from the configure.ac; they could just supply that. Not that that seems to > be doing anything less doing any less pointless. It's text, but it is an intermediate or "object" file. It's not doing pointless stuff; it's coping with the myriad platforms and variants that Python has support for. And did you even read my previous post? You need autoconf to generate it; without configure, you need another step in the build AND another program installed. So the object file is committed to remove a dependency. > It's like calling dis.dis() on a >> >> Python function and then complaining that it's long and unreadable. >> >> Stop making unfair criticisms if you want people to take you seriously. > > > My point about systems so complex that you can't understand the whole thing > still stands, and it applies to a lot of things I come across. > > The CPython source is just a topical example. Do /you/ know for sure that > there is no dead weight in there? Or do you just accept it because, after > all, the thing works [for you], and it doesn't matter if it's bigger and > slower than it needs to be. You're welcome to go through CPython to find unnecessary code if you like. Just read the SOURCE CODE, not the intermediate files. Stop exaggerating the situation. ChrisA From invalid at invalid.invalid Sat Feb 17 20:18:09 2018 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 18 Feb 2018 01:18:09 +0000 (UTC) Subject: Gmane seems to be gone Message-ID: I've been dreading this moment for a couple years: it looks like gmane.org is gone. The original operator/maintainer gave up a couple years ago and pulled the plug. Somebody else took over at that point. The Web UI was never revived, but the basic NNTP<->mailing-list gateway continue to work -- until tonight. Now the domain is gone. Perhaps it's just an oversight, but I've got a bad feeling... I really, really prefer using slrn and an NNTP server to follow mailing lists, and I was using gmane to follow about a dozen of them (including the Python list). At least for python, there's still a Usenet gateway. Unfortunately, that's not true for any of the other lists that I used to follow on gmane.org. Does anybody have any idea what it would take to set up a private NNTP server that served articles from a dozen or so IMAP mailboxes? How hard would it be to write something like that in Python? I've got a pretty good handle on the IMAP end of things, but I've got no clue about the NNTP server end. -- Grant From bc at freeuk.com Sat Feb 17 20:31:13 2018 From: bc at freeuk.com (bartc) Date: Sun, 18 Feb 2018 01:31:13 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 18/02/2018 00:45, Chris Angelico wrote: > On Sun, Feb 18, 2018 at 11:13 AM, bartc wrote: > It's text, but it is an intermediate or "object" file. It's not doing > pointless stuff; it's coping with the myriad platforms and variants > that Python has support for. It could well do all that. But it surely cannot need 18,000 lines' worth to do it; that much should be obvious to anyone. And in fact, for building with MS's Visual Studio, it doesn't use that file at all, but something smaller. (Although the MS build adds its own complexities.) Anyway a lot of that stuff also seems to be going on inside the C header files - have a look inside pyconfig.h. (I tried my own C compiler on it and it seems to think it's MSVC.) This part of it is incredibly messy and could do with a rewrite. > You're welcome to go through CPython to find unnecessary code if you > like. Just read the SOURCE CODE, not the intermediate files. Stop > exaggerating the situation. I'm sorry, I must have misunderstood your post where you said that by reducing the size of a program and spreading it over fewer files, you found it easier to detect problems like redundant code. CPython comprises 660 .h and .c files (looking only at C sources). Not the biggest project (I believe gcc has 45,000 such files), but still a fair number. And there are 4000 files in all. -- bartc From rosuav at gmail.com Sat Feb 17 20:39:27 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Feb 2018 12:39:27 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On Sun, Feb 18, 2018 at 12:31 PM, bartc wrote: > On 18/02/2018 00:45, Chris Angelico wrote: >> >> On Sun, Feb 18, 2018 at 11:13 AM, bartc wrote: > > >> It's text, but it is an intermediate or "object" file. It's not doing >> pointless stuff; it's coping with the myriad platforms and variants >> that Python has support for. > > > It could well do all that. But it surely cannot need 18,000 lines' worth to > do it; that much should be obvious to anyone. And in fact, for building with > MS's Visual Studio, it doesn't use that file at all, but something smaller. > (Although the MS build adds its own complexities.) You're arguing against a strawman, and you know it. Either that, or you have a fundamental misunderstanding of "intermediate files", in which case I recommend you spend some time with a web search engine. I'm done. ChrisA From torriem at gmail.com Sat Feb 17 23:59:13 2018 From: torriem at gmail.com (Michael Torrie) Date: Sat, 17 Feb 2018 21:59:13 -0700 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: <73ae2ea4-efc4-994f-a094-25456212a9cb@gmail.com> On 02/17/2018 06:31 PM, bartc wrote: > It could well do all that. But it surely cannot need 18,000 lines' worth > to do it; that much should be obvious to anyone. And in fact, for > building with MS's Visual Studio, it doesn't use that file at all, but > something smaller. (Although the MS build adds its own complexities.) Hardly. Sorry Bart, but you're barking up the wrong tree there. There are may legitimate complaints about autoconf, but being able to read the generated configure script isn't one of them. That's like saying the disassembly of a compiled executable is needlessly complex. Of all people you should understand that, having made your own compilers. Managing a cross-platform build is very complicated stuff. Autoconf works, although there are other systems that work well too. I suspect you'd complain about them also, as they also generate scripts and makefiles that you'd find difficult to read. > Anyway a lot of that stuff also seems to be going on inside the C header > files - have a look inside pyconfig.h. (I tried my own C compiler on it > and it seems to think it's MSVC.) I'm pretty sure pyconfig.h is generated by the build system. So spending a lot of time in there isn't really that useful, except as a way of determining what features and configurations will be used by the rest of the source code. If you think Python's C header files are complicated, you should look at some of the GNU standard library header files sometime. Or really any cross-platform library's header files. As I said, managing the configuration of source code across platforms is a large and complicated problem. From breamoreboy at gmail.com Sun Feb 18 02:14:09 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 17 Feb 2018 23:14:09 -0800 (PST) Subject: Gmane seems to be gone In-Reply-To: References: Message-ID: <7a4364bf-47f3-44e3-a707-9db4f876204d@googlegroups.com> On Sunday, February 18, 2018 at 1:18:20 AM UTC, Grant Edwards wrote: > I've been dreading this moment for a couple years: it looks like > gmane.org is gone. The original operator/maintainer gave up a couple > years ago and pulled the plug. Somebody else took over at that point. > The Web UI was never revived, but the basic NNTP<->mailing-list gateway > continue to work -- until tonight. Now the domain is gone. Perhaps > it's just an oversight, but I've got a bad feeling... > > I really, really prefer using slrn and an NNTP server to follow > mailing lists, and I was using gmane to follow about a dozen of them > (including the Python list). At least for python, there's still a > Usenet gateway. Unfortunately, that's not true for any of the other > lists that I used to follow on gmane.org. > > Does anybody have any idea what it would take to set up a private NNTP > server that served articles from a dozen or so IMAP mailboxes? > > How hard would it be to write something like that in Python? I've got > a pretty good handle on the IMAP end of things, but I've got no clue > about the NNTP server end. > > -- > Grant I can still see it. I subscribe to gwene.com.stackoverflow.feeds.julia-lang and I received the last two questions at 02:05 and 03:33 UTC. I recall that when things used to go up the creek on the Python side of things I'd email larsi at gnus dot org and he'd restart things. I've no idea what to do nowadays, sorry. -- Kindest regards. Mark Lawrence. From cl at isbd.net Sun Feb 18 04:11:24 2018 From: cl at isbd.net (Chris Green) Date: Sun, 18 Feb 2018 09:11:24 +0000 Subject: Gmane seems to be gone References: Message-ID: Grant Edwards wrote: > I've been dreading this moment for a couple years: it looks like > gmane.org is gone. The original operator/maintainer gave up a couple > years ago and pulled the plug. Somebody else took over at that point. > The Web UI was never revived, but the basic NNTP<->mailing-list gateway > continue to work -- until tonight. Now the domain is gone. Perhaps > it's just an oversight, but I've got a bad feeling... > I think it was a short term hiccough, a posting of mine got bounced yesterday but subsequent ones worked OK. -- Chris Green ? From marko at pacujo.net Sun Feb 18 04:16:29 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 18 Feb 2018 11:16:29 +0200 Subject: Gmane seems to be gone References: Message-ID: <871shisln6.fsf@elektro.pacujo.net> Grant Edwards : > Does anybody have any idea what it would take to set up a private NNTP > server that served articles from a dozen or so IMAP mailboxes? > > How hard would it be to write something like that in Python? I've got > a pretty good handle on the IMAP end of things, but I've got no clue > about the NNTP server end. I bet NNTP () is a simpler beast than IMAP. Maybe Twisted has some support for it, I don't know, but writing a server from scratch wouldn't be too bad, either. However, the simplest way forward might be to just take an off-the-shelf NNTP server and write a IMAP/NNTP gateway bot that acts as a client bothways. Then you can use Python's nntplib and imaplib. Marko From martin.schoon at gmail.com Sun Feb 18 04:28:59 2018 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 18 Feb 2018 09:28:59 GMT Subject: Mayavi website? References: Message-ID: Den 2018-02-17 skrev Martin Sch??n : > Anyone else having problems with interacting with > http://code.enthought.com/pages/mayavi-project.html > ? > Later yesterday I found this: http://docs.enthought.com/mayavi/mayavi/ and it works without a hitch. /Martin From linux4michelle at tamay-dogan.net Sun Feb 18 04:50:57 2018 From: linux4michelle at tamay-dogan.net (Michelle Konzack) Date: Sun, 18 Feb 2018 11:50:57 +0200 Subject: Gmane seems to be gone In-Reply-To: References: Message-ID: <5feaeb7016c66caa8995a3f2b71990ac.squirrel@webmail.tamay-dogan.net> The Whois Registry was updated today for the next 12 month. Am 2018-02-18 hackte Grant Edwards in die Tasten: > I've been dreading this moment for a couple years: it looks like > gmane.org is gone. The original operator/maintainer gave up a couple > years ago and pulled the plug. Somebody else took over at that point. > The Web UI was never revived, but the basic NNTP<->mailing-list gateway > continue to work -- until tonight. Now the domain is gone. Perhaps > it's just an oversight, but I've got a bad feeling... > > I really, really prefer using slrn and an NNTP server to follow > mailing lists, and I was using gmane to follow about a dozen of them > (including the Python list). At least for python, there's still a > Usenet gateway. Unfortunately, that's not true for any of the other > lists that I used to follow on gmane.org. > > Does anybody have any idea what it would take to set up a private NNTP > server that served articles from a dozen or so IMAP mailboxes? > > How hard would it be to write something like that in Python? I've got > a pretty good handle on the IMAP end of things, but I've got no clue > about the NNTP server end. Thanks in advance -- Michelle Konzack Miila ITSystems @ TDnet GNU/Linux Developer 00372-54541400 From rosuav at gmail.com Sun Feb 18 04:51:18 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Feb 2018 20:51:18 +1100 Subject: Python 2 to 3 Conversion In-Reply-To: References: Message-ID: On Sun, Feb 18, 2018 at 4:35 AM, Wildman via Python-list wrote: > Thanks to Chris and Ben. Your suggestions were slightly > different but both worked equally well, although I don't > understand how that can be so. > >> struct.pack('256s', ifname[:15].encode('ascii')) >> struct.pack('256s', ifname.encode('ascii')) Those two will be identical for short interface names, but the first form will truncate a longer name before encoding. I don't know what the ioctl will do with a really long ifname, so it's probably worth hanging onto the [:15] slicing. > I was looking for a reliable way to determine the IP addy > for a given network adapter. That is what the code does > but I don't understand how it does it either. > > Thanks again. I will continue to research and study the > code in the hope I will understand it. Ah, makes sense. I'd probably do something like this: >>> import socket >>> s = socket.socket(type=socket.SOCK_DGRAM) >>> s.connect(("8.8.8.8", 53)) >>> s.getsockname()[0] '192.168.0.19' But that's only going to show one (uplink) address. If I needed to get ALL addresses for ALL network adapters, I'd either look for a library, and if one wasn't easily found, I'd shell out to the "ip" command and parse its output. :) ChrisA From rosuav at gmail.com Sun Feb 18 04:57:02 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Feb 2018 20:57:02 +1100 Subject: Python on Android? Message-ID: Does anyone have experience with running Python scripts on Android phones? I have a brother (honestly! I'm not actually using a phone myself!) who's trying to run one of my scripts in QPython, which claims to be version 3.2.2. I think that really truly is a Python 3.2 implementation - probing for newer features suggests that it actually doesn't even support the u"..." syntax that came (back) in with Python 3.3. So... does anyone know of a Python interpreter that's compatible with 3.4 or better and runs on Android? Personal experiences trump just searching the web... ChrisA From linux4michelle at tamay-dogan.net Sun Feb 18 04:57:42 2018 From: linux4michelle at tamay-dogan.net (Michelle Konzack) Date: Sun, 18 Feb 2018 11:57:42 +0200 Subject: Gmane seems to be gone In-Reply-To: References: Message-ID: Hi Grant, Am 2018-02-18 hackte Grant Edwards in die Tasten: > Does anybody have any idea what it would take to set up a private NNTP > server that served articles from a dozen or so IMAP mailboxes? Where is the problem? I have my own Root-Server am subscribed with 32 mail accounts to arround 120 mailinglists every mailaccount is a group like "Debian", "Devel", ... I use the courier-imap Server which allow me to setup shared folders... Which mean, any user of my mailserver can add this shared folders to his own account! Hence, he has a copy of the Mailinglist... ;-) and of course, it works with any decent IMAP clients. However, under Windows there are a bunch of problems (lake of capabilities) Have a nice Sunday -- Michelle Konzack Miila ITSystems @ TDnet GNU/Linux Developer 00372-54541400 From cl at isbd.net Sun Feb 18 05:30:52 2018 From: cl at isbd.net (Chris Green) Date: Sun, 18 Feb 2018 10:30:52 +0000 Subject: Gmane seems to be gone References: <871shisln6.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > Grant Edwards : > > > Does anybody have any idea what it would take to set up a private NNTP > > server that served articles from a dozen or so IMAP mailboxes? > > > > How hard would it be to write something like that in Python? I've got > > a pretty good handle on the IMAP end of things, but I've got no clue > > about the NNTP server end. > > I bet NNTP () is a simpler > beast than IMAP. > > Maybe Twisted has some support for it, I don't know, but writing a > server from scratch wouldn't be too bad, either. > > However, the simplest way forward might be to just take an off-the-shelf > NNTP server and write a IMAP/NNTP gateway bot that acts as a client > bothways. Then you can use Python's nntplib and imaplib. > Leafnode might be a good place to start. -- Chris Green ? From bc at freeuk.com Sun Feb 18 06:33:29 2018 From: bc at freeuk.com (bartc) Date: Sun, 18 Feb 2018 11:33:29 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 18/02/2018 01:39, Chris Angelico wrote: > On Sun, Feb 18, 2018 at 12:31 PM, bartc wrote: >> On 18/02/2018 00:45, Chris Angelico wrote: >>> >>> On Sun, Feb 18, 2018 at 11:13 AM, bartc wrote: >> >> >>> It's text, but it is an intermediate or "object" file. It's not doing >>> pointless stuff; it's coping with the myriad platforms and variants >>> that Python has support for. >> >> >> It could well do all that. But it surely cannot need 18,000 lines' worth to >> do it; that much should be obvious to anyone. And in fact, for building with >> MS's Visual Studio, it doesn't use that file at all, but something smaller. >> (Although the MS build adds its own complexities.) > > You're arguing against a strawman, and you know it. Either that, or > you have a fundamental misunderstanding of "intermediate files", in > which case I recommend you spend some time with a web search engine. > > I'm done. Then you have your head in the sand. Whether you are looking at the 18000-line file, or the 5000-line one, you should be asking yourself what on earth any of that gobbledygook has to do with compiling CPython? What files does it generate or modify, if any; what is the end result of it? According to the diagram here: https://en.wikipedia.org/wiki/Configure_script, the eventual output is a config.h file, and a makefile. Hey, why don't they just provide those files?! (Sometimes projects do just that, provide a range of makefiles for example.) Then it just needs 'make', which is a tool that comes with the C compiler that is needed anyway. (However, I don't even use 'make'; look at the arrow between the 'make' process and the executable: implied here are invocations of tools such as compilers and linkers. This is where I usually work. This is all that is usually needed.) -- bartc From breamoreboy at gmail.com Sun Feb 18 06:39:11 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 18 Feb 2018 03:39:11 -0800 (PST) Subject: Gmane seems to be gone In-Reply-To: References: Message-ID: On Sunday, February 18, 2018 at 9:16:26 AM UTC, Chris Green wrote: > Grant Edwards wrote: > > I've been dreading this moment for a couple years: it looks like > > gmane.org is gone. The original operator/maintainer gave up a couple > > years ago and pulled the plug. Somebody else took over at that point. > > The Web UI was never revived, but the basic NNTP<->mailing-list gateway > > continue to work -- until tonight. Now the domain is gone. Perhaps > > it's just an oversight, but I've got a bad feeling... > > > I think it was a short term hiccough, a posting of mine got bounced > yesterday but subsequent ones worked OK. > > -- > Chris Green > ? Nothing from this thread is showing on gmane and nothing from the gnucash users' mailing list has come through since around 18:00 17/02/2018 UTC. I've just sent an email to the gmane.discuss list asking them to take a look, got an "Authorization required" response which I duly replied to and promptly got "Delivery Status Notification (Failure)", the guts of which are "DNS Error: 2475639 DNS type 'mx' lookup of auth.gmane.org responded with code NXDOMAIN Domain name not found: auth.gmane.org". Bah humbug :( -- Kindest regards. Mark Lawrence. From ned at nedbatchelder.com Sun Feb 18 06:45:42 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sun, 18 Feb 2018 06:45:42 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: On 2/18/18 6:33 AM, bartc wrote: > On 18/02/2018 01:39, Chris Angelico wrote: >> On Sun, Feb 18, 2018 at 12:31 PM, bartc wrote: >>> On 18/02/2018 00:45, Chris Angelico wrote: >>>> >>>> On Sun, Feb 18, 2018 at 11:13 AM, bartc wrote: >>> >>> >>>> It's text, but it is an intermediate or "object" file. It's not doing >>>> pointless stuff; it's coping with the myriad platforms and variants >>>> that Python has support for. >>> >>> >>> It could well do all that. But it surely cannot need 18,000 lines' >>> worth to >>> do it; that much should be obvious to anyone. And in fact, for >>> building with >>> MS's Visual Studio, it doesn't use that file at all, but something >>> smaller. >>> (Although the MS build adds its own complexities.) >> >> You're arguing against a strawman, and you know it. Either that, or >> you have a fundamental misunderstanding of "intermediate files", in >> which case I recommend you spend some time with a web search engine. >> >> I'm done. > > Then you have your head in the sand. Whether you are looking at the > 18000-line file, or the 5000-line one, you should be asking yourself > what on earth any of that gobbledygook has to do with compiling CPython? > > What files does it generate or modify, if any; what is the end result > of it? > > According to the diagram here: > https://en.wikipedia.org/wiki/Configure_script, the eventual output is > a config.h file, and a makefile. Hey, why don't they just provide > those files?! (Sometimes projects do just that, provide a range of > makefiles for example.) > > Then it just needs 'make', which is a tool that comes with the C > compiler that is needed anyway. > > (However, I don't even use 'make'; look at the arrow between the > 'make' process and the executable: implied here are invocations of > tools such as compilers and linkers. This is where I usually work. > This is all that is usually needed.) > Let's not go down this path yet again.? We've heard it all before. Bart: stop it.? Everyone else: stop it. :) --Ned. From wegge at wegge.dk Sun Feb 18 06:55:59 2018 From: wegge at wegge.dk (Anders Wegge Keller) Date: Sun, 18 Feb 2018 12:55:59 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <85zi48b6r5.fsf@benfinney.id.au> References: <85zi48b6r5.fsf@benfinney.id.au> Message-ID: <20180218125559.64d31761@wegge.dk> P? Sat, 17 Feb 2018 15:05:34 +1100 Ben Finney skrev: > boB Stepp writes: > He blithely conflates ?weakly typed? (Python objects are not weakly, but > very strongly typed) Python is more strongly typed than PHP, but that doesn't really say much. However, compared to a language like C, there are big holes in the type safety. >>> alist = [1, 'two', ('three', four), 5*'#'] That list is not only weakly typed, but rather untyped. There are no safeguards in the language, that enforce that all elements in a list or other container are in fact of the same type. Before type annotations and mypy, I could not enforce that other than at runtime. So claiming Python to have very strongly typed objects is a bit of a strecth. PHP that many of us like to be smug about is equally strongly typed. It just comes with a metric ton of inane type coersions that have gone too far. However, we are happy to add an integer to a float without having to make an explicit conversion every time, so ... Thread carefull and all that. -- //Wegge From bc at freeuk.com Sun Feb 18 06:57:47 2018 From: bc at freeuk.com (bartc) Date: Sun, 18 Feb 2018 11:57:47 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: <%SdiC.70902$fC6.19731@fx24.am4> On 18/02/2018 11:45, Ned Batchelder wrote: > Let's not go down this path yet again.? We've heard it all before. Bart: > stop it.? Everyone else: stop it. :) Well, this was a rare instance of someone admitting that a simple and smaller codebase has benefits in being able to detect problems such as redundancy. I'm taking the opportunity to say that that can apply in many other situations too. -- bartc From ned at nedbatchelder.com Sun Feb 18 07:18:52 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sun, 18 Feb 2018 07:18:52 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <%SdiC.70902$fC6.19731@fx24.am4> References: <%SdiC.70902$fC6.19731@fx24.am4> Message-ID: On 2/18/18 6:57 AM, bartc wrote: > On 18/02/2018 11:45, Ned Batchelder wrote: > >> Let's not go down this path yet again. We've heard it all before. >> Bart: stop it.? Everyone else: stop it. :) > > Well, this was a rare instance of someone admitting that a simple and > smaller codebase has benefits in being able to detect problems such as > redundancy. > > I'm taking the opportunity to say that that can apply in many other > situations too. > We've heard you, thanks. --Ned. From Richard at Damon-Family.org Sun Feb 18 07:34:03 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 18 Feb 2018 07:34:03 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <20180218125559.64d31761@wegge.dk> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> Message-ID: <3ef54e6f-c66e-725f-767a-f8342b19704e@Damon-Family.org> On 2/18/18 6:55 AM, Anders Wegge Keller wrote: > P? Sat, 17 Feb 2018 15:05:34 +1100 > Ben Finney skrev: >> boB Stepp writes: > >> He blithely conflates ?weakly typed? (Python objects are not weakly, but >> very strongly typed) > Python is more strongly typed than PHP, but that doesn't really say much. > However, compared to a language like C, there are big holes in the type > safety. > > >>> alist = [1, 'two', ('three', four), 5*'#'] > > That list is not only weakly typed, but rather untyped. There are no > safeguards in the language, that enforce that all elements in a list or > other container are in fact of the same type. Before type annotations and > mypy, I could not enforce that other than at runtime. > > So claiming Python to have very strongly typed objects is a bit of a > strecth. PHP that many of us like to be smug about is equally strongly > typed. It just comes with a metric ton of inane type coersions that have > gone too far. However, we are happy to add an integer to a float without > having to make an explicit conversion every time, so ... Thread carefull and > all that. > Note, that 'Only at Runtime' isn't a distinction between Weakly Typed or Strongly Typed but between Statically Typed and Dynamically Typed. Python is clearly dynamically typed, as we don't need to assign a type to a symbol, and it will always reflect a value of that type, Python is much stronger typed than PHP, because in PHP you can do things like 1 + '2' and get 3, as string values will naturally convert themselves to numbers, Python won't do this. Yes Python will freely convert between numeric types, but I wouldn't say that Python claims to be a language that focuses on numerics. -- Richard Damon From wegge at wegge.dk Sun Feb 18 07:45:39 2018 From: wegge at wegge.dk (Anders Wegge Keller) Date: Sun, 18 Feb 2018 13:45:39 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <3ef54e6f-c66e-725f-767a-f8342b19704e@Damon-Family.org> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <3ef54e6f-c66e-725f-767a-f8342b19704e@Damon-Family.org> Message-ID: <20180218134539.47ecfda4@wegge.dk> P? Sun, 18 Feb 2018 07:34:03 -0500 Richard Damon skrev: > Python is much stronger typed than PHP, because in PHP you can do things > like 1 + '2' and get 3, as string values will naturally convert > themselves to numbers, Python won't do this. Yes Python will freely > convert between numeric types, but I wouldn't say that Python claims to > be a language that focuses on numerics. Type coercion is not the same as weak typing. Let me stress that the difference between languages having 1 + 1.23 as a valid construct and languages having 1 + '1.23' as a valid construct, is merely a design descision. PHP is fully aware that a string and an integer have different types, and makes a implicit cast, rather than throw an error. -- //Wegge From arj.python at gmail.com Sun Feb 18 08:16:13 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 18 Feb 2018 17:16:13 +0400 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: have the impression he is not a python coder Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ On 17 Feb 2018 07:22, "boB Stepp" wrote: > This article is written by Nathan Murthy, a staff software engineer at > Tesla. The article is found at: > https://medium.com/@natemurthy/all-the-things-i- > hate-about-python-5c5ff5fda95e > > Apparently he chose his article title as "click bait". Apparently he > does not really hate Python (So he says.). His leader paragraph is: > > "Python is viewed as a ubiquitous programming language; however, its > design limits its potential as a reliable and high performance systems > language. Unfortunately, not every developer is aware of its > limitations." > > As I currently do not have the necessary technical knowledge to > properly evaluate his claims, I thought I would ask those of you who > do. I have neither the knowledge or boB-hours to write a large > distributed system code base, but I am curious if Python is truly > limited for doing these in the ways he claims. > > BTW, I am not trying to start (another) heated, emotional thread. You > guys do sometimes get carried away! I honestly just what to know the > truth of the matters out of my continuing to learn Python. I suspect > there is probably some truth in his claims, but I am not sure if he is > taking things out of their proper application contexts or not. > > Thanks! > -- > boB > -- > https://mail.python.org/mailman/listinfo/python-list > From brozi at toya.net.pl Sun Feb 18 08:39:59 2018 From: brozi at toya.net.pl (A.Brozi) Date: Sun, 18 Feb 2018 14:39:59 +0100 Subject: solve_ivp problem (scipy 1.0.0) Message-ID: <2175be55-859c-23b9-4189-1c9a9e28729f@toya.net.pl> Hello In new "scipy" (1.0.0) I've found procedure "solve_ivp", which makes it possible to use "events" in order to terminate the ode solution when some condition is satisfied. The precise moment of termination (the value of independent variable) is to be found in "t_events", but I cannot find the dependent variable(s) value(s) at the termination moment. Am I missing something? Or it's simply not possible (hopefully 'not yet')? Regards Andrzej Brozi From torriem at gmail.com Sun Feb 18 10:44:39 2018 From: torriem at gmail.com (Michael Torrie) Date: Sun, 18 Feb 2018 08:44:39 -0700 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <20180218134539.47ecfda4@wegge.dk> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <3ef54e6f-c66e-725f-767a-f8342b19704e@Damon-Family.org> <20180218134539.47ecfda4@wegge.dk> Message-ID: <567e6887-e081-6818-d204-adaaec732e34@gmail.com> On 02/18/2018 05:45 AM, Anders Wegge Keller wrote: > P? Sun, 18 Feb 2018 07:34:03 -0500 > Richard Damon skrev: > >> Python is much stronger typed than PHP, because in PHP you can do things >> like 1 + '2' and get 3, as string values will naturally convert >> themselves to numbers, Python won't do this. Yes Python will freely >> convert between numeric types, but I wouldn't say that Python claims to >> be a language that focuses on numerics. > > Type coercion is not the same as weak typing. Let me stress that the > difference between languages having 1 + 1.23 as a valid construct and > languages having 1 + '1.23' as a valid construct, is merely a design > descision. PHP is fully aware that a string and an integer have different > types, and makes a implicit cast, rather than throw an error. And indeed language design decisions bring about weak or strong typing. So it's not a stretch at all to say Python has strongly typed objects. It does. Once created, objects are what they are; they cannot be changed into different objects without creating a new object. As for numeric coercion, Python could have made numeric types stronger and dropped all coercion. Maybe it should have. But it appears to generally follow the same coercion patters as other commonly-used static type languages like C. In this we can agree that Python's typing was weakened, but so also is C. It's interesting to see C++ move to become more like a dynamically-typed language like Python. I recently saw a talk about how C++ inheritance is not as good as we all thought it was. Why should it matter what an object inherits from, if it supports the interface we desire? (Duck typing, essentially). So there's now a rather complicated C++17 idiom for wrapping objects with special interfaces so that we can make a list that contains different types (just like python!) that happen to all support the same method signature. Anyway, was interesting. From best_lay at yahoo.com Sun Feb 18 11:18:45 2018 From: best_lay at yahoo.com (Wildman) Date: Sun, 18 Feb 2018 10:18:45 -0600 Subject: Python 2 to 3 Conversion References: Message-ID: On Sun, 18 Feb 2018 20:51:18 +1100, Chris Angelico wrote: > On Sun, Feb 18, 2018 at 4:35 AM, Wildman via Python-list > wrote: >> Thanks to Chris and Ben. Your suggestions were slightly >> different but both worked equally well, although I don't >> understand how that can be so. >> >>> struct.pack('256s', ifname[:15].encode('ascii')) >>> struct.pack('256s', ifname.encode('ascii')) > > Those two will be identical for short interface names, but the first > form will truncate a longer name before encoding. I don't know what > the ioctl will do with a really long ifname, so it's probably worth > hanging onto the [:15] slicing. Since the interfaces are named by the (Linux) system, I don't think long interface names will be a problem. Names are used like lo, eth0, wlan0, ppp0, vmnet8, etc. But, I will keep the [:15} slicing just in case. >> I was looking for a reliable way to determine the IP addy >> for a given network adapter. That is what the code does >> but I don't understand how it does it either. >> >> Thanks again. I will continue to research and study the >> code in the hope I will understand it. > > Ah, makes sense. I'd probably do something like this: > >>>> import socket >>>> s = socket.socket(type=socket.SOCK_DGRAM) >>>> s.connect(("8.8.8.8", 53)) >>>> s.getsockname()[0] > '192.168.0.19' Brief background... I am working on a project that reports Linux system info and lan is part of it. To start with I was using code similar to the above but, with the help of a few testers in some of the Linux newsgroups, I found it did not work correctly in all situations such as 'live' sessions. So I started looking for an alternative. > But that's only going to show one (uplink) address. If I needed to get > ALL addresses for ALL network adapters, I'd either look for a library, > and if one wasn't easily found, I'd shell out to the "ip" command and > parse its output. :) > > ChrisA I considered using the "ip" command but I prefer not to depend on external programs if I can get around it. I know that might be considered silly but that's just me. Thanks for your input. -- GNU/Linux user #557453 The cow died so I don't need your bull! From mailman at hanez.org Sun Feb 18 11:57:34 2018 From: mailman at hanez.org (Johannes Findeisen) Date: Sun, 18 Feb 2018 17:57:34 +0100 Subject: Python on Android? In-Reply-To: References: Message-ID: <20180218175734.6e8908d9@orbit.fritz.box> On Sun, 18 Feb 2018 20:57:02 +1100 Chris Angelico wrote: > Does anyone have experience with running Python scripts on Android > phones? I have a brother (honestly! I'm not actually using a phone > myself!) who's trying to run one of my scripts in QPython, which > claims to be version 3.2.2. I think that really truly is a Python 3.2 > implementation - probing for newer features suggests that it actually > doesn't even support the u"..." syntax that came (back) in with Python > 3.3. So... does anyone know of a Python interpreter that's compatible > with 3.4 or better and runs on Android? > There is an App for Android called "Pydroid 3". You can find it in the Google Play Store [0]. It provides a Python interpreter in version 3.6.2 in its current release. The Python binary is installed under /data/user/0/ru.iiec.pydroid3/files/arm-linux-androideabi/bin/python but I can not access it in a normal terminal without being user "root" but it is usable from within the terminal included in the Pydroid App. I do only execute some basic Python stuff for fun and learning when traveling but that should work as you can open and save your code from the integrated editor. Johannes [0] https://play.google.com/store/apps/details?id=ru.iiec.pydroid3 From invalid at invalid.invalid Sun Feb 18 12:22:05 2018 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 18 Feb 2018 17:22:05 +0000 (UTC) Subject: Gmane seems to be gone References: Message-ID: On 2018-02-18, Chris Green wrote: > Grant Edwards wrote: >> I've been dreading this moment for a couple years: it looks like >> gmane.org is gone. The original operator/maintainer gave up a couple >> years ago and pulled the plug. Somebody else took over at that point. >> The Web UI was never revived, but the basic NNTP<->mailing-list gateway >> continue to work -- until tonight. Now the domain is gone. Perhaps >> it's just an oversight, but I've got a bad feeling... >> > I think it was a short term hiccough, a posting of mine got bounced > yesterday but subsequent ones worked OK. Does news.gmane.org (or any gmane.org domain) resolve for you? It doesn't using Google's public DNS server (8.8.8.8). The whois database at icann.org says the gmane.org domain registration was updated today, and it now belongs to Yomura Corporation of Atlanta, Georga. -- Grant From invalid at invalid.invalid Sun Feb 18 12:25:18 2018 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 18 Feb 2018 17:25:18 +0000 (UTC) Subject: Gmane seems to be gone References: Message-ID: On 2018-02-18, Michelle Konzack wrote: > Am 2018-02-18 hackte Grant Edwards in die Tasten: > >> Does anybody have any idea what it would take to set up a private NNTP >> server that served articles from a dozen or so IMAP mailboxes? > > Where is the problem? > > I have my own Root-Server am subscribed with 32 mail accounts to arround > 120 mailinglists every mailaccount is a group like "Debian", "Devel", ... > > I use the courier-imap Server which allow me to setup shared folders... > Which mean, any user of my mailserver can add this shared folders to his > own account! > > Hence, he has a copy of the Mailinglist... The problem is that I want to use an NNTP clinet, not an IMAP client. I find NNTP clients are much better suited for following high-volume lists. -- Grant From invalid at invalid.invalid Sun Feb 18 12:26:54 2018 From: invalid at invalid.invalid (Grant Edwards) Date: Sun, 18 Feb 2018 17:26:54 +0000 (UTC) Subject: Gmane seems to be gone References: Message-ID: On 2018-02-18, Grant Edwards wrote: > On 2018-02-18, Chris Green wrote: >> Grant Edwards wrote: >>> I've been dreading this moment for a couple years: it looks like >>> gmane.org is gone. The original operator/maintainer gave up a couple >>> years ago and pulled the plug. Somebody else took over at that point. >>> The Web UI was never revived, but the basic NNTP<->mailing-list gateway >>> continue to work -- until tonight. Now the domain is gone. Perhaps >>> it's just an oversight, but I've got a bad feeling... >>> >> I think it was a short term hiccough, a posting of mine got bounced >> yesterday but subsequent ones worked OK. > > Does news.gmane.org (or any gmane.org domain) resolve for you? > > It doesn't using Google's public DNS server (8.8.8.8). > > The whois database at icann.org says the gmane.org domain registration > was updated today, and it now belongs to Yomura Corporation of > Atlanta, Georga. It was Yomura who picked up the archive and continued the gateway service a couple years ago, and they just renewed the domain for another year, so perhaps it's just a temporary DNS problem? -- Grant From kirillbalunov at gmail.com Sun Feb 18 12:29:02 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Sun, 18 Feb 2018 20:29:02 +0300 Subject: Python on Android? In-Reply-To: <20180218175734.6e8908d9@orbit.fritz.box> References: <20180218175734.6e8908d9@orbit.fritz.box> Message-ID: To mention another way to use Python on Android is Termux project . From their site - Termux is an *Android terminal emulator and Linux environment app* that works directly with no rooting or setup required and Python is a side-effect. There are still some restrictions but in general you can install everything you want. I even ran Jupyter notebook, but on the phone it is certainly a mockery :) With kine regards, -gdg 2018-02-18 19:57 GMT+03:00 Johannes Findeisen : > On Sun, 18 Feb 2018 20:57:02 +1100 > Chris Angelico wrote: > > > Does anyone have experience with running Python scripts on Android > > phones? I have a brother (honestly! I'm not actually using a phone > > myself!) who's trying to run one of my scripts in QPython, which > > claims to be version 3.2.2. I think that really truly is a Python 3.2 > > implementation - probing for newer features suggests that it actually > > doesn't even support the u"..." syntax that came (back) in with Python > > 3.3. So... does anyone know of a Python interpreter that's compatible > > with 3.4 or better and runs on Android? > > > > There is an App for Android called "Pydroid 3". You can find it in > the Google Play Store [0]. It provides a Python interpreter in version > 3.6.2 in its current release. > > The Python binary is installed under > > /data/user/0/ru.iiec.pydroid3/files/arm-linux-androideabi/bin/python > > but I can not access it in a normal terminal without being user "root" > but it is usable from within the terminal included in the Pydroid App. > > I do only execute some basic Python stuff for fun and learning when > traveling but that should work as you can open and save your code from > the integrated editor. > > Johannes > > [0] https://play.google.com/store/apps/details?id=ru.iiec.pydroid3 > -- > https://mail.python.org/mailman/listinfo/python-list > From python at mrabarnett.plus.com Sun Feb 18 12:48:33 2018 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 18 Feb 2018 17:48:33 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <20180218125559.64d31761@wegge.dk> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> Message-ID: <688b22ba-3104-b98b-3a84-4f7d9ce278ea@mrabarnett.plus.com> On 2018-02-18 11:55, Anders Wegge Keller wrote: > P? Sat, 17 Feb 2018 15:05:34 +1100 > Ben Finney skrev: >> boB Stepp writes: > > >> He blithely conflates ?weakly typed? (Python objects are not weakly, but >> very strongly typed) > > Python is more strongly typed than PHP, but that doesn't really say much. > However, compared to a language like C, there are big holes in the type > safety. > > >>> alist = [1, 'two', ('three', four), 5*'#'] > > That list is not only weakly typed, but rather untyped. There are no > safeguards in the language, that enforce that all elements in a list or > other container are in fact of the same type. Before type annotations and > mypy, I could not enforce that other than at runtime. > That's not untyped. If you want to see what untyped is, look at BCPL, which has only the "bit-pattern". In an untyped language, operators and functions take values (bit-patterns) and return values, so there would be one operator for integer addition and another operator for floating-point addition. If you do integer addition on a value that represents a floating-point number, it won't complain, but the result will be meaningless. Similarly, it won't complain if you use a floating-point number, consisting of a mantissa and exponent, as a pointer. > So claiming Python to have very strongly typed objects is a bit of a > strecth. PHP that many of us like to be smug about is equally strongly > typed. It just comes with a metric ton of inane type coersions that have > gone too far. However, we are happy to add an integer to a float without > having to make an explicit conversion every time, so ... Thread carefull and > all that. > From python at mrabarnett.plus.com Sun Feb 18 12:51:46 2018 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 18 Feb 2018 17:51:46 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <567e6887-e081-6818-d204-adaaec732e34@gmail.com> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <3ef54e6f-c66e-725f-767a-f8342b19704e@Damon-Family.org> <20180218134539.47ecfda4@wegge.dk> <567e6887-e081-6818-d204-adaaec732e34@gmail.com> Message-ID: On 2018-02-18 15:44, Michael Torrie wrote: > On 02/18/2018 05:45 AM, Anders Wegge Keller wrote: >> P? Sun, 18 Feb 2018 07:34:03 -0500 >> Richard Damon skrev: >> >>> Python is much stronger typed than PHP, because in PHP you can do things >>> like 1 + '2' and get 3, as string values will naturally convert >>> themselves to numbers, Python won't do this. Yes Python will freely >>> convert between numeric types, but I wouldn't say that Python claims to >>> be a language that focuses on numerics. >> >> Type coercion is not the same as weak typing. Let me stress that the >> difference between languages having 1 + 1.23 as a valid construct and >> languages having 1 + '1.23' as a valid construct, is merely a design >> descision. PHP is fully aware that a string and an integer have different >> types, and makes a implicit cast, rather than throw an error. > > And indeed language design decisions bring about weak or strong typing. > So it's not a stretch at all to say Python has strongly typed objects. > It does. Once created, objects are what they are; they cannot be changed > into different objects without creating a new object. > > As for numeric coercion, Python could have made numeric types stronger > and dropped all coercion. Maybe it should have. But it appears to > generally follow the same coercion patters as other commonly-used static > type languages like C. In this we can agree that Python's typing was > weakened, but so also is C. > Some languages, such as Pascal, allow "widening", where an integer can be coerced to a real (float), but not the other way round. C _does_ let you coerce a double to a float (rounding) and both to an int (truncating). > It's interesting to see C++ move to become more like a dynamically-typed > language like Python. I recently saw a talk about how C++ inheritance > is not as good as we all thought it was. Why should it matter what an > object inherits from, if it supports the interface we desire? (Duck > typing, essentially). So there's now a rather complicated C++17 idiom > for wrapping objects with special interfaces so that we can make a list > that contains different types (just like python!) that happen to all > support the same method signature. Anyway, was interesting. > From python at mrabarnett.plus.com Sun Feb 18 12:55:09 2018 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 18 Feb 2018 17:55:09 +0000 Subject: Python on Android? In-Reply-To: References: Message-ID: <419a2e5d-bdbb-5cbc-d4d5-9f5619f89edd@mrabarnett.plus.com> On 2018-02-18 09:57, Chris Angelico wrote: > Does anyone have experience with running Python scripts on Android > phones? I have a brother (honestly! I'm not actually using a phone > myself!) who's trying to run one of my scripts in QPython, which > claims to be version 3.2.2. I think that really truly is a Python 3.2 > implementation - probing for newer features suggests that it actually > doesn't even support the u"..." syntax that came (back) in with Python > 3.3. So... does anyone know of a Python interpreter that's compatible > with 3.4 or better and runs on Android? > > Personal experiences trump just searching the web... > I've tried running Python on a Kindle Fire HD, but it lacked some encodings, so I don't know how complete the implementation is. From grant.b.edwards at gmail.com Sun Feb 18 13:03:14 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 18 Feb 2018 18:03:14 +0000 (UTC) Subject: Gmane seems to be gone References: Message-ID: On 2018-02-18, Dennis Lee Bieber wrote: > On Sun, 18 Feb 2018 17:26:54 +0000 (UTC), Grant Edwards > declaimed the following: > >> >>It was Yomura who picked up the archive and continued the gateway >>service a couple years ago, and they just renewed the domain for >>another year, so perhaps it's just a temporary DNS problem? > > Looks like the DNS update is propagating -- I can now reach gmane; but > it hasn't shown any new traffic on any of the groups/lists I follow. Yay! Gmane.org just started resolving again, and I'm posting this followup from slrn via news.gmane.org. I really should donate some money to gmane's operators... -- Grant From mailman at hanez.org Sun Feb 18 13:06:23 2018 From: mailman at hanez.org (Johannes Findeisen) Date: Sun, 18 Feb 2018 19:06:23 +0100 Subject: Python on Android? In-Reply-To: References: <20180218175734.6e8908d9@orbit.fritz.box> Message-ID: <20180218190623.166307c2@orbit.fritz.box> On Sun, 18 Feb 2018 20:29:02 +0300 Kirill Balunov wrote: > To mention another way to use Python on Android is Termux project > . From their site - Termux is an *Android terminal > emulator and Linux environment app* that works directly with no rooting or > setup required and Python is a side-effect. There are still some > restrictions but in general you can install everything you want. I even ran > Jupyter notebook, but on the phone it is certainly a mockery :) Wow, that project looks great. But, if I only wanna play around with Python it seems a little bit oversized. Anyway it seems to become one of my most used apps on my phone when traveling. I for long time looked for something like this because the use of the default Android terminal utilities really is a pain. Very nice suggestion. Thanks for that! Another way I used Python on Android was Kivy [0]. But it is very different because it is made for building GUI applications for Android using Python. But the nice thing is, that you can package your Python scripts into a native APK containing an own Python inteerpreter. So, think of, building a small application where you can bind your Python code to some buttons and execute it when clicking. Not very useful when you need the output but very useful when you have some tasks you wanna execute with just one click... ;) Johannes [0] http://kivy.org From arj.python at gmail.com Sun Feb 18 13:47:36 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 18 Feb 2018 22:47:36 +0400 Subject: Fwd: Re: Python on Android? In-Reply-To: References: Message-ID: Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ ---------- Forwarded message ---------- From: "Abdur-Rahmaan Janhangeer" Date: 18 Feb 2018 14:17 Subject: Re: Python on Android? To: "Chris Angelico" Cc: mailed the developer sometimes back, told me to download a 3.4 included .apk on the github repo Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ On 18 Feb 2018 13:59, "Chris Angelico" wrote: > Does anyone have experience with running Python scripts on Android > phones? I have a brother (honestly! I'm not actually using a phone > myself!) who's trying to run one of my scripts in QPython, which > claims to be version 3.2.2. I think that really truly is a Python 3.2 > implementation - probing for newer features suggests that it actually > doesn't even support the u"..." syntax that came (back) in with Python > 3.3. So... does anyone know of a Python interpreter that's compatible > with 3.4 or better and runs on Android? > > Personal experiences trump just searching the web... > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From cl at isbd.net Sun Feb 18 14:26:36 2018 From: cl at isbd.net (Chris Green) Date: Sun, 18 Feb 2018 19:26:36 +0000 Subject: Gmane seems to be gone References: Message-ID: Grant Edwards wrote: > On 2018-02-18, Chris Green wrote: > > Grant Edwards wrote: > >> I've been dreading this moment for a couple years: it looks like > >> gmane.org is gone. The original operator/maintainer gave up a couple > >> years ago and pulled the plug. Somebody else took over at that point. > >> The Web UI was never revived, but the basic NNTP<->mailing-list gateway > >> continue to work -- until tonight. Now the domain is gone. Perhaps > >> it's just an oversight, but I've got a bad feeling... > >> > > I think it was a short term hiccough, a posting of mine got bounced > > yesterday but subsequent ones worked OK. > > Does news.gmane.org (or any gmane.org domain) resolve for you? > chris$ host gmane.org gmane.org has address 104.31.85.1 gmane.org has address 104.31.84.1 gmane.org has IPv6 address 2400:cb00:2048:1::681f:5401 gmane.org has IPv6 address 2400:cb00:2048:1::681f:5501 gmane.org mail is handled by 10 mxfilter-048034.atla03.us.yomura.com. gmane.org mail is handled by 10 mxfilter-048035.atla03.us.yomura.com. chris$ host news.gmane.org news.gmane.org is an alias for blaine.gmane.org. blaine.gmane.org has address 195.159.176.226 blaine.gmane.org mail is handled by 5 blaine.gmane.org. chris$ -- Chris Green ? From breamoreboy at gmail.com Sun Feb 18 15:06:35 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Sun, 18 Feb 2018 20:06:35 +0000 Subject: Gmane seems to be gone In-Reply-To: References: Message-ID: On 18/02/18 18:03, Grant Edwards wrote: > On 2018-02-18, Dennis Lee Bieber wrote: >> On Sun, 18 Feb 2018 17:26:54 +0000 (UTC), Grant Edwards >> declaimed the following: >> >>> >>> It was Yomura who picked up the archive and continued the gateway >>> service a couple years ago, and they just renewed the domain for >>> another year, so perhaps it's just a temporary DNS problem? >> >> Looks like the DNS update is propagating -- I can now reach gmane; but >> it hasn't shown any new traffic on any of the groups/lists I follow. > > Yay! > > Gmane.org just started resolving again, and I'm posting this followup > from slrn via news.gmane.org. I really should donate some money to > gmane's operators... > Great stuff :) Of course as gmane wasn't working I posted earlier today on gg but I believe you've opted out of that so wouldn't have seen it :( What the hell, normal service is resumed :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From marko at pacujo.net Sun Feb 18 15:08:21 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 18 Feb 2018 22:08:21 +0200 Subject: Gmane seems to be gone References: Message-ID: <87tvueqcwa.fsf@elektro.pacujo.net> Grant Edwards : > The problem is that I want to use an NNTP clinet, not an IMAP client. > I find NNTP clients are much better suited for following high-volume > lists. Well, GNUS makes *all* email look like NNTP. For example, email messages expire after grace period by default. Marko From p.f.moore at gmail.com Sun Feb 18 15:22:06 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 18 Feb 2018 20:22:06 +0000 Subject: Fw: [issue22167] iglob() has misleading documentation (does indeed store names internally) In-Reply-To: <20180217074718.0b9dfd05@bigbox.christie.dr> References: <20180217074718.0b9dfd05@bigbox.christie.dr> Message-ID: You're shown as nosy on that issue: 2014-08-08 01:24:52 Gumnos set nosy: + Gumnos Paul On 17 February 2018 at 13:47, Tim Chase wrote: > Has anybody else been getting unexpected/unsolicited emails from the > Python bug-tracker? > > I'm not associated with (didn't submit/lurk/follow/sign-up-for) this > bug or its notifications but somehow I'm getting messages on this > particular issue. I've now received two notifications (both on this > same bug) coming to my custom Python Bugs email address. When I > check the issue's page, I'm not on the Nosy list so I don't really > have a good way to unfollow because it doesn't think I'm following it. > > I *am* in the Python Bugs DB associated with other issues, I'm just > confused how I ended up attached to this bug. Anybody else either > receiving unsolicited bug notifications or happen to know how/why I'm > getting these? > > Thanks, > > -tkc > > > Begin forwarded message: > > Date: Sat, 17 Feb 2018 08:09:23 +0000 > From: Serhiy Storchaka > To: python.bugs@[redacted] > Subject: [issue22167] iglob() has misleading documentation (does > indeed store names internally) > > > Serhiy Storchaka added the comment: > > Unfortunately issue25596 didn't change anything about this issue. > iglob() still stores names (actually DirEntry objects) of all files > in a directory before starting yielding the first of them. Otherwise > we cold exceed the limit of open file descriptors. > > ---------- > nosy: +serhiy.storchaka > > _______________________________________ > Python tracker > > _______________________________________ > -- > https://mail.python.org/mailman/listinfo/python-list From timothy.c.delaney at gmail.com Sun Feb 18 16:47:14 2018 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Mon, 19 Feb 2018 08:47:14 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <20180218125559.64d31761@wegge.dk> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> Message-ID: On 18 February 2018 at 22:55, Anders Wegge Keller wrote: > P? Sat, 17 Feb 2018 15:05:34 +1100 > Ben Finney skrev: > > boB Stepp writes: > > > > He blithely conflates ?weakly typed? (Python objects are not weakly, but > > very strongly typed) > > Python is more strongly typed than PHP, but that doesn't really say much. > However, compared to a language like C, there are big holes in the type > safety. > > >>> alist = [1, 'two', ('three', four), 5*'#'] > > That list is not only weakly typed, but rather untyped. There are no > safeguards in the language, that enforce that all elements in a list or > other container are in fact of the same type. Before type annotations and > mypy, I could not enforce that other than at runtime. > You couldn't have got the above much more wrong. As others have said, typing is about how the underlying memory is treated. I can't comment on PHP typing, as I've actively avoided that language since my first experience with it. C is statically and weakly typed. Variables know their types at compile time (static typing). It is a feature of the language that you can cast any pointer to any chunk of memory to be a pointer to any other type (normally via void *). This is not coercion - it takes the bit pattern of memory of one type and interprets it as the bit pattern for another type, and is weak typing. Python is strongly and dynamically typed. In Python, once you create an object, it remains that type of object, no matter what you do to it*. That makes it strongly typed. Python does not have variables - it instead has names with no type information at compile time. That makes it dynamically typed. In your list example, each element of the list is a name - the element itself doesn't have a type, but the object named by the list does. * In some cases it is possible to change the __class__ of an object, but that can only be done in restricted circumstances and will usually result in runtime exceptions unless you've specifically planned your class hierarchy to do it. The cases where it is possible to change the __class__ do not result in reinterpretation of memory bit patterns. >>> (1.0).__class__ = int Traceback (most recent call last): File "", line 1, in TypeError: __class__ assignment only supported for heap types or ModuleType subclasses In some implementations it is possible to subvert the Python typing system by stepping out of Python code and into (for example) a C extension, but that does not make Python *the language* weakly typed. Tim Delaney From steve+comp.lang.python at pearwood.info Sun Feb 18 19:09:40 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 00:09:40 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: Message-ID: On Sun, 18 Feb 2018 08:02:10 +1100, Chris Angelico wrote: [...] >> The usual response to that is to make ever-finer-grained types, until >> the type-system can prove the code is correct. >> >> integers >> positive integers >> positive integers greater than 10 >> positive integers greater than 10 but less than 15003 >> positive odd integers greater than 10 but less than 15003 >> positive odd integers greater than 10 but less than 15003 divisible >> by 17 >> >> Of course, this has a few minor (ha!) difficulties... starting with the >> hardest problem in computer science, naming things. > > Naming things isn't a problem if we're working with a type inference > system. It is as soon as you need to name a type in order to document it, talk about it to other human beings, or declare it for the type system. (Type inference can only go so far -- you do sometimes need to give it hints.) > On the flip side, if your last example is purely type inference, > it's not really a type checking system - it's a holistic static > analysis. You can't say "TypeError: spamminess must be less than 15003" > without also saying "oh but that might be a bug in this function, since > it's meant to be able to take numbers >= 15003". That's *always* a problem with type systems, regardless of whether type inference is used or not. If I declare that a function takes a list, then the type checker will complain if I pass it a tuple. That's what type checkers do. But they do it even if the algorithm (or source code!) is perfectly capable of using the same steps for working with both lists and tuples. In older languages like Pascal you have to write something like this to satisfy the compiler: # using Python syntax for symplicity def listtail(x: list): return x[1:] def tupletail(x: tuple): return x[1:] then remember to call listtail(alist) and tupletail(atuple) as appropriate. This meaningless make-work to satisfy the compiler is one of the biggest problems with older static type-systems, and is one of the things that object oriented programming is intended to solve. More modern statically typed languages may include some form of type- dispatch, where you still have to write the tail function twice: def tail(x:list): return x[1:] def tail(x:tuple): return x[1:] but now you use the same name for both and the compiler can automatically use the right one. Another modern solution is the idea of explicit union types, so you can say that x must be a list *or* a tuple. An alternate solution is to give up on the static typing and use dynamic typing, which in turn allows us to introduce duck-typing as a strategy. > Some of the type names CAN be generated algebraically. Sure, but only the most boring, uninteresting kinds of types can be so named. The point is that "sufficiently fine-grained types" can be arbitrarily complex. If a human finds it hard to give it a meaningful name, no algorithm will do it either. Consider: "positive odd integers greater than 10 but less than 15003 divisible by 17 except for 850, 867 and 1394; or primes that aren't Mersenne primes". Or more realistically, suppose you want your type system to ensure that you don't lay off the wrong workers: "employee records where the length of employment is greater than six months but less than five years, AND the salary is greater than $100,000 or less than $40,000, AND the average number of paid sick days taken per year over the last three years is less than five, AND the number of reprimands is zero in the last year and less than two in the last five years, AND the position in the last two stack rankings was in the top fifty percent" and so on. It is not practical for a type system to be "sufficiently fine- grained": that would require the type system to be as flexible and Turing complete as the language itself (and consequently as bug-ridden as the rest of our code). We cannot expect the type system to find interesting bugs, like flagging the best performers for layoff instead of the worst. It will mostly find only boring bugs like trying to add 1 + "one". [...] > Oh wait. We're talking about programmers here. Malicious has nothing on > the rampant stupidity... > > https://thedailywtf.com/articles/What_Is_Truth_0x3f_ :-) -- Steve From ben+python at benfinney.id.au Sun Feb 18 20:16:06 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 19 Feb 2018 12:16:06 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? References: <017k8dp5aohn4fagp00tp3f22v90jrk764@4ax.com> Message-ID: <85po51bwyx.fsf@benfinney.id.au> Dennis Lee Bieber writes: > On Mon, 19 Feb 2018 00:09:40 +0000 (UTC), Steven D'Aprano > declaimed the following: > > >Or more realistically, suppose you want your type system to ensure > >that you don't lay off the wrong workers: > > > >"employee records where the length of employment is greater than six > >months but less than five years, AND the salary is greater than > >$100,000 or less than $40,000, AND the average number of paid sick > >days taken per year over the last three years is less than five, AND > >the number of reprimands is zero in the last year and less than two > >in the last five years, AND the position in the last two stack > >rankings was in the top fifty percent" > > > That's not a data type -- that's an SQL query... If that isn't part of the definition of the data type, then *the type checker is not going to help* to determine whether the code lays off the wrong workers. > The data type is still "employee" I think you're agreeing with Steven. Static data types are not relevant for finding most of the bugs a program can have. -- \ ?The most dangerous man to any government is the man who is | `\ able to think things out for himself, without regard to the | _o__) prevailing superstitions and taboos.? ?Henry L. Mencken | Ben Finney From Richard at Damon-Family.org Sun Feb 18 20:40:54 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 18 Feb 2018 20:40:54 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <85po51bwyx.fsf@benfinney.id.au> References: <017k8dp5aohn4fagp00tp3f22v90jrk764@4ax.com> <85po51bwyx.fsf@benfinney.id.au> Message-ID: <6d8ec3bc-fcb1-feb7-8be1-3b9a93bd5c41@Damon-Family.org> On 2/18/18 8:16 PM, Ben Finney wrote: > Dennis Lee Bieber writes: > >> On Mon, 19 Feb 2018 00:09:40 +0000 (UTC), Steven D'Aprano >> declaimed the following: >> >>> Or more realistically, suppose you want your type system to ensure >>> that you don't lay off the wrong workers: >>> >>> "employee records where the length of employment is greater than six >>> months but less than five years, AND the salary is greater than >>> $100,000 or less than $40,000, AND the average number of paid sick >>> days taken per year over the last three years is less than five, AND >>> the number of reprimands is zero in the last year and less than two >>> in the last five years, AND the position in the last two stack >>> rankings was in the top fifty percent" >>> >> That's not a data type -- that's an SQL query... > If that isn't part of the definition of the data type, then *the type > checker is not going to help* to determine whether the code lays off the > wrong workers. Doesn't sound like the sort of thing that you should expect a Type Checker to find. Now, perhaps you could define a type of something like 'Layoffable_Employee' with an invariant based on the above, then the layoff module could insist that it will only that an Layoffable_Employee, and the type uses pre-/post- conditions to confirm the needed conditions. Now, somewhere in the code you probably started with an Employee, and then to pass it to the module you will need to convert the type of the object (it seems unlikely that such a type is inherent in most of the rest of the code). This is why it doesn't make sense for it to be a 'Type', but more likely an attribute/predicate of the object, that the Layoff module can confirm as a pre-condition. > >> The data type is still "employee" > I think you're agreeing with Steven. Static data types are not relevant > for finding most of the bugs a program can have. > -- Richard Damon From ben+python at benfinney.id.au Sun Feb 18 20:54:06 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 19 Feb 2018 12:54:06 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? References: <017k8dp5aohn4fagp00tp3f22v90jrk764@4ax.com> <85po51bwyx.fsf@benfinney.id.au> <6d8ec3bc-fcb1-feb7-8be1-3b9a93bd5c41@Damon-Family.org> Message-ID: <85lgfpbv7l.fsf@benfinney.id.au> Richard Damon writes: > Doesn't sound like the sort of thing that you should expect a Type > Checker to find. Yes, exactly. This point is evidently ignored by people who moan that Python, though it has strong types, is prone to bugs because it has dynamic binding. As Steven points out, and you say so succinctly, there are plenty of bugs a programmer will write, that a type checker can't catch. So dynamic binding is not a barrier to catching those bugs. -- \ ?Listen: we are here on Earth to fart around. Don't let anybody | `\ tell you otherwise.? ?_Timequake_, Kurt Vonnegut | _o__) | Ben Finney From bc at freeuk.com Sun Feb 18 21:14:26 2018 From: bc at freeuk.com (bartc) Date: Mon, 19 Feb 2018 02:14:26 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: <4qqiC.223282$kD1.96803@fx34.am4> On 19/02/2018 00:09, Steven D'Aprano wrote: > Sure, but only the most boring, uninteresting kinds of types can be so > named. The point is that "sufficiently fine-grained types" can be > arbitrarily complex. I don't think so. If a human finds it hard to give it a meaningful > name, no algorithm will do it either. Consider: > > "positive odd integers greater than 10 but less than 15003 divisible by > 17 except for 850, 867 and 1394; or primes that aren't Mersenne primes". Is that a type? Or a function? Or a set? Or a constraint? How would even a type for the odd numbers from 1 to 10 inclusive work? (That, a type consisting of one of the values in {1,3,5,7,9}.) Would they be ordered or unordered? Can I do arithmetic with them: will 3*3 work, but not 3*5? This is where keeping things simple pays off. -- bartc From rosuav at gmail.com Sun Feb 18 21:59:09 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 19 Feb 2018 13:59:09 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <4qqiC.223282$kD1.96803@fx34.am4> References: <4qqiC.223282$kD1.96803@fx34.am4> Message-ID: On Mon, Feb 19, 2018 at 1:14 PM, bartc wrote: > On 19/02/2018 00:09, Steven D'Aprano wrote: > >> Sure, but only the most boring, uninteresting kinds of types can be so >> named. The point is that "sufficiently fine-grained types" can be >> arbitrarily complex. > > > I don't think so. > > If a human finds it hard to give it a meaningful >> >> name, no algorithm will do it either. Consider: >> >> "positive odd integers greater than 10 but less than 15003 divisible by >> 17 except for 850, 867 and 1394; or primes that aren't Mersenne primes". > > > Is that a type? Or a function? Or a set? Or a constraint? > > How would even a type for the odd numbers from 1 to 10 inclusive work? > (That, a type consisting of one of the values in {1,3,5,7,9}.) Would they be > ordered or unordered? Can I do arithmetic with them: will 3*3 work, but not > 3*5? The type is "positive odd number below ten" and could be written as int(1..9|1%2). That is an orderable type; you can say that 3 < 7, for instance. And yes, arithmetic would be defined just fine; there's no requirement for the result of an operation to have the same type as its inputs: >>> 5 / 2 # two integers 2.5 I don't understand what your concern here is. ChrisA From wegge at wegge.dk Sun Feb 18 22:26:32 2018 From: wegge at wegge.dk (Anders Wegge Keller) Date: Mon, 19 Feb 2018 04:26:32 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> Message-ID: <20180219042632.79afa5b3@wegge.dk> P? Mon, 19 Feb 2018 08:47:14 +1100 Tim Delaney skrev: > On 18 February 2018 at 22:55, Anders Wegge Keller wrote: > > That list is not only weakly typed, but rather untyped. There are no > > safeguards in the language, that enforce that all elements in a list or > > other container are in fact of the same type. Before type annotations and > > mypy, I could not enforce that other than at runtime. > You couldn't have got the above much more wrong. > As others have said, typing is about how the underlying memory is treated. And that is exactly my point. Python does not have a typed list. It have a list that takes whatever is thrown into it. I'll skip the rest, as you totally missed the point. -- //Wegge From steve+comp.lang.python at pearwood.info Sun Feb 18 23:10:28 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 04:10:28 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <017k8dp5aohn4fagp00tp3f22v90jrk764@4ax.com> Message-ID: On Sun, 18 Feb 2018 19:38:03 -0500, Dennis Lee Bieber wrote: > On Mon, 19 Feb 2018 00:09:40 +0000 (UTC), Steven D'Aprano > declaimed the following: > > >>Or more realistically, suppose you want your type system to ensure that >>you don't lay off the wrong workers: >> >>"employee records where the length of employment is greater than six >>months but less than five years, AND the salary is greater than $100,000 >>or less than $40,000, AND the average number of paid sick days taken per >>year over the last three years is less than five, AND the number of >>reprimands is zero in the last year and less than two in the last five >>years, AND the position in the last two stack rankings was in the top >>fifty percent" >> > That's not a data type -- that's an SQL query... > > The data type is still "employee" Dennis, you've missed the context. We're discussing the claim that given a sufficiently fine-grained type- system, type checkers can prove any program is correct. Under that scenario, we need separate types for integers, positive integers, negative-or-positive-integers-but-not-zero, employees, employees with seniority, employees with reprimands on their file, employees with three or more reprimands on their file, etc. Every tiny distinction between data values becomes a type, and (so the claim goes) the type checker can prove that your program is 100% bug free. You will never accidentally fire the wrong person, because the compiler won't let you call file_employee(x) if x is the wrong type of employee. Of course its nonsense, but that's the claim I'm discussing. -- Steve From steve+comp.lang.python at pearwood.info Sun Feb 18 23:32:04 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 04:32:04 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> Message-ID: On Mon, 19 Feb 2018 02:14:26 +0000, bartc wrote: > On 19/02/2018 00:09, Steven D'Aprano wrote: > >> Sure, but only the most boring, uninteresting kinds of types can be so >> named. The point is that "sufficiently fine-grained types" can be >> arbitrarily complex. > > I don't think so. That's nice. Do you have any reasons for your position? >> If a human finds it hard to give it a meaningful >> name, no algorithm will do it either. Consider: >> >> "positive odd integers greater than 10 but less than 15003 divisible by >> 17 except for 850, 867 and 1394; or primes that aren't Mersenne >> primes". > > Is that a type? Or a function? Or a set? Or a constraint? It *could* be a type, if your type system was sufficiently flexible to allow you to specify something in that level of detail. Of course no existing type system is. Which is of course my point: why static typing zealots do like to make grandiose claims about "sufficiently fine-grained types", the reality is that no type system is, or can be, sufficiently fine-grained to avoid all runtime validation. > How would even a type for the odd numbers from 1 to 10 inclusive work? That's an easy one: even Pascal in the 1970s could deal with enumerated types like the values 1, 3, 5, 7, 9. (I think.) So that's something that a type checker could easily verify at compile time. Given: declare x, y : OddIntBetweenOneAndTen; then: x := 3 would be allowed, but: y := x + 2 probably would be rejected, because the compiler may not be able to tell whether or not x + 2 is still an OddIntBetweenOneAndTen (unless it is actually tracking the value of x, which type checkers don't normally do). Of course this put *severe* limitations on what you can do with such highly restrictive types, which is also part of my point. > (That, a type consisting of one of the values in {1,3,5,7,9}.) Would > they be ordered or unordered? Can I do arithmetic with them: will 3*3 > work, but not 3*5? Operations on types do not necessarily have to return their same type. There's nothing wrong with saying that Odd plus Odd returns Even. -- Steve From steve+comp.lang.python at pearwood.info Sun Feb 18 23:39:31 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 04:39:31 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> Message-ID: On Mon, 19 Feb 2018 04:26:32 +0100, Anders Wegge Keller wrote: > P? Mon, 19 Feb 2018 08:47:14 +1100 > Tim Delaney skrev: >> On 18 February 2018 at 22:55, Anders Wegge Keller >> wrote: > > >> > That list is not only weakly typed, but rather untyped. There are no >> > safeguards in the language, that enforce that all elements in a list >> > or other container are in fact of the same type. Before type >> > annotations and mypy, I could not enforce that other than at runtime. > >> You couldn't have got the above much more wrong. > >> As others have said, typing is about how the underlying memory is >> treated. > > And that is exactly my point. Python does not have a typed list. It > have a list that takes whatever is thrown into it. > > I'll skip the rest, as you totally missed the point. I think its actually you have totally missed the point. What you want is a homogeneous list, a list which only accepts items of the same type. The built-in list is not that data structure, but Python already has something similar: see the array module. If you want a list type which only supports items of a single type, you can get one by subclassing. But that's utterly irrelevant to the question of strong versus weak typing. A strongly typed language like Haskell, which does (almost?) no automatic coercions at all (possibly not even int to float) can still define a "list of Any object" type, just like Python's built-in list. That doesn't make Haskell untyped, and it is a sign of your confusion that you think it makes Python untyped. Sorry to be blunt, but you're not even in the right ballpark here. Python is strongly typed because a list is a list, regardless of its contents. If you say "1 + []" Python does not try to cast or convert the list into an int and add them, or interpret the int as a list, and perform list concatenation on a chunk of memory that is actually an int but is being treated as an (almost certainly invalid!) list. Remember: - the static/dynamic distinction is between *when* types are checked (compile time or run time), not *whether* they are checked; - untyped languages treat *everything* (or almost everything) as the same type: usually either a chunk of raw bits, like in assembly, or text strings, like in many shell scripting languages; - weakly typed languages do a lot of automatic coercions; - strongly typed languages do very few or no automatic coercions. None of those distinctions have anything to say about whether the standard library offers a homogeneous list where all the items have the same type. -- Steve From alain at universite-de-strasbourg.fr.invalid Mon Feb 19 03:40:09 2018 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Mon, 19 Feb 2018 09:40:09 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> Message-ID: <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Tim Delaney writes: [...] > As others have said, typing is about how the underlying memory is treated. No. It is much more than that. Typing is about everything you can say about a given statement. Some type systems are focusing on type labels only (like most statically typed programming languages), others are fairly elaborate (for instance, Eiffel's, if requires/ensures are considered part of the type system). There is a wide range of type systems, more or less practical depending on your needs (see https://www.cis.upenn.edu/~bcpierce/tapl/ if you want to see how far you can go with typing). [...] > C is statically and weakly typed. Variables know their types at compile > time (static typing). It is a feature of the language that you can cast any > pointer to any chunk of memory to be a pointer to any other type (normally > via void *). This is not coercion - it takes the bit pattern of memory of > one type and interprets it as the bit pattern for another type, and is weak > typing. No. C has much stronger rules, not on casting, but on accessing the pointees, which basically invalidates your argument. Refer to the C standard for details. > Python is strongly and dynamically typed. In Python, once you create an > object, it remains that type of object, no matter what you do to it*. That > makes it strongly typed. [...] But you can modify the class (not __class__) in whatever way you want. For instance: class X(object): def f(self): ... ... del X.f So, in Python, knowing that object x is an instance of class X tells you... essentially nothing. Call this strong typing if you want. In terms of type systems, it is (strong) simplistic-typing based on type labels, and labels carry no information whatsoever. Dynamic typing lets you avoid doing nonsense with bit-patterns, as you say, by gracefully crashing at run-time instead of performing "unexpected" actions. "Unexpected" is defined by the whole *execution* of the program up to the point where typing takes place. To understand whether a Python statement x.f() is meaningful (which is what typing is about), you have to know the full sequence of actions that have taken place on x *and* on its class (and on everything that that call may use) since the beginning of the program. For any non-trivial program, this is usually way too complex to capture, testing will be incomplete, and all you can do is run your program and see whether is goes through. As a general rule, if that's all you expect from typing, then, fine, call this "strong". I won't go as far, and just say that it is good enough for the programs I use Python for. -- Alain. From sjsumitj at gmail.com Mon Feb 19 03:57:23 2018 From: sjsumitj at gmail.com (Sum J) Date: Mon, 19 Feb 2018 14:27:23 +0530 Subject: How to reset system proxy using pyhton code Message-ID: Hi, I am using below python code (Python 2.7) to reset the proxy of my Ubuntu (Cent OS 6) system, but I am unable to reset the proxy: Code : import os print "Unsetting http..." os.system("unset http_proxy") os.system("echo $http_proxy") print "http is reset" Output : Unsetting http... http://web-proxy.xxxx.xxxxxxx.net:8080 http is reset Process finished with exit code 0 It should not return ' http://web-proxy.xxxx.xxxxxxx.net:8080 ' in output. I run the same unset command from terminal , then I see that proxy is reset: [trex at sumlnxvm ~]$ unset $HTTP_PROXY [trex at sumlnxvm ~]$ echo $HTTP_PROXY [trex at sumlnxvm ~]$ Please suggest how to reset system proxy using Python Code Regards, Sumit From rosuav at gmail.com Mon Feb 19 04:14:32 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 19 Feb 2018 20:14:32 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Mon, Feb 19, 2018 at 7:40 PM, Alain Ketterlin wrote: > Tim Delaney writes: >> C is statically and weakly typed. Variables know their types at compile >> time (static typing). It is a feature of the language that you can cast any >> pointer to any chunk of memory to be a pointer to any other type (normally >> via void *). This is not coercion - it takes the bit pattern of memory of >> one type and interprets it as the bit pattern for another type, and is weak >> typing. > > No. C has much stronger rules, not on casting, but on accessing the > pointees, which basically invalidates your argument. Refer to the C > standard for details. Really? What rules? $ cat demo.c; gcc -Wall demo.c; ./a.out #include int main() { float f = 3.14159; int *i = (int *)&f; printf("As an integer, %f is %d\n", f, *i); return 0; } As an integer, 3.141590 is 1078530000 $ Looks to me like C is perfectly happy to interpret a float as an int. What rules are you seeing violated here? > But you can modify the class (not __class__) in whatever way you want. > For instance: > > class X(object): > def f(self): ... > ... > del X.f > > So, in Python, knowing that object x is an instance of class X tells > you... essentially nothing. Call this strong typing if you want. In > terms of type systems, it is (strong) simplistic-typing based on type > labels, and labels carry no information whatsoever. Sure you can. And you can do a lot of other things at run time, too. Monkey-patching doesn't change the fact that x really and truly is an instance of class X, it just changes what you can do with that object. So what you're saying is that Python's dynamism makes it possible to disrupt duck typing. Sure, I'll grant you that. But the type system itself isn't broken by that. ChrisA From antoon.pardon at vub.be Mon Feb 19 04:36:20 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Mon, 19 Feb 2018 10:36:20 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: <8af6bb82-fef7-96d0-279c-037568b81aa1@vub.be> On 17-02-18 21:11, Chris Angelico wrote: > On Sun, Feb 18, 2018 at 1:47 AM, Ian Kelly wrote: >> On Fri, Feb 16, 2018 at 9:32 PM, Chris Angelico wrote: >>> You'd be surprised how rarely that kind of performance even matters. >>> The author of that article cites C# as a superior language, but in the >>> rewrite from C# to Python (the same one I mentioned in the other >>> post), I sped the program up incredibly. Part of that is because C# >>> requires the startup of its framework (in my case that's Mono) just as >>> Python does, and partly it's because the simplicity of Python let me >>> eliminate a number of unnecessary HTTP requests. Trust me, your choice >>> of language doesn't help you if it means you do three (sequential) >>> HTTP requests when one would have done. Clean code has its own >>> advantages. >> Okay, I'm curious. How did C# force you to make extra HTTP requests >> that were no longer necessary when you rewrote in Python? > It didn't *force* those requests to be made, but the code was so large > and convoluted that I doubt its original author realized that the > requests were being repeated. Pseudo-coded: But was that a reflection on C# or on the original author? -- Antoon Pardon From tjol at tjol.eu Mon Feb 19 04:46:27 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 19 Feb 2018 10:46:27 +0100 Subject: How to reset system proxy using pyhton code In-Reply-To: References: Message-ID: <2c4a6626-c6b4-9c4d-9210-01fee82e96a2@tjol.eu> On 2018-02-19 09:57, Sum J wrote: > Hi, > > I am using below python code (Python 2.7) to reset the proxy of my Ubuntu > (Cent OS 6) system, but I am unable to reset the proxy: I'm sure you know this, but CentOS and Ubuntu are two different things. > > Code : > import os > print "Unsetting http..." > os.system("unset http_proxy") > os.system("echo $http_proxy") > print "http is reset" Please pay attention to indentation when pasting Python code. I know what you mean, but as such this code wouldn't even run. > > Output : > Unsetting http... > http://web-proxy.xxxx.xxxxxxx.net:8080 > http is reset > Process finished with exit code 0 > > It should not return ' http://web-proxy.xxxx.xxxxxxx.net:8080 ' in output. > > I run the same unset command from terminal , then I see that proxy is > reset: > > [trex at sumlnxvm ~]$ unset $HTTP_PROXY > [trex at sumlnxvm ~]$ echo $HTTP_PROXY It is not possible to modify a parent process's environment from within a child process. Can I interest you in a shell function or alias? If you want to remove the environment variable for future processes you start from your python script, simply modify os.environ. For example: (using an environment variable I actually have on my system) % cat del_env.py import os os.system('echo desktop $XDG_SESSION_DESKTOP') print('removing') del os.environ['XDG_SESSION_DESKTOP'] os.system('echo desktop $XDG_SESSION_DESKTOP') % python del_env.py desktop gnome removing desktop % echo $XDG_SESSION_DESKTOP gnome -- Thomas > > [trex at sumlnxvm ~]$ > > > Please suggest how to reset system proxy using Python Code > > Regards, > Sumit > From tjreedy at udel.edu Mon Feb 19 04:56:23 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 19 Feb 2018 04:56:23 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On 2/19/2018 4:14 AM, Chris Angelico wrote: > On Mon, Feb 19, 2018 at 7:40 PM, Alain Ketterlin > wrote: >> Tim Delaney writes: >>> C is statically and weakly typed. Variables know their types at compile >>> time (static typing). It is a feature of the language that you can cast any >>> pointer to any chunk of memory to be a pointer to any other type (normally >>> via void *). This is not coercion - it takes the bit pattern of memory of >>> one type and interprets it as the bit pattern for another type, and is weak >>> typing. >> >> No. C has much stronger rules, not on casting, but on accessing the >> pointees, which basically invalidates your argument. Refer to the C >> standard for details. > > Really? What rules? > > $ cat demo.c; gcc -Wall demo.c; ./a.out > #include > > int main() { > float f = 3.14159; > int *i = (int *)&f; > printf("As an integer, %f is %d\n", f, *i); > return 0; > } > > As an integer, 3.141590 is 1078530000 > $ > > Looks to me like C is perfectly happy to interpret a float as an int. > What rules are you seeing violated here? The last time I tried, C was also willing to ints and arrays thereof as an array of chars. (I vaguely remember Fortran doing something like this also. One Fortran int = four chars.) The C memory model is a linear sequence of bytes, each consisting of at least 8 bits. One means for malware attacks it is to disguise and insert binary code bytes as character bytes and then trick the CPU into executing the 'characters'. -- Terry Jan Reedy From tjol at tjol.eu Mon Feb 19 05:02:31 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 19 Feb 2018 11:02:31 +0100 Subject: solve_ivp problem (scipy 1.0.0) In-Reply-To: <2175be55-859c-23b9-4189-1c9a9e28729f@toya.net.pl> References: <2175be55-859c-23b9-4189-1c9a9e28729f@toya.net.pl> Message-ID: On 2018-02-18 14:39, A.Brozi wrote: > Hello > > In new "scipy" (1.0.0) I've found procedure "solve_ivp", which makes it > possible to use "events" in order to terminate the ode solution when > some condition is satisfied. > The precise moment of termination (the value of independent variable) is > to be found in "t_events", but I cannot find the dependent variable(s) > value(s) at the termination moment. > Am I missing something? Or it's simply not possible (hopefully 'not yet')? > > Regards > Andrzej Brozi Isn't the solution passed back as ?y?? -- Thomas From alain at universite-de-strasbourg.fr.invalid Mon Feb 19 05:04:47 2018 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Mon, 19 Feb 2018 11:04:47 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: <87a7w5uwg0.fsf@universite-de-strasbourg.fr.invalid> Chris Angelico writes: > On Mon, Feb 19, 2018 at 7:40 PM, Alain Ketterlin > wrote: >> No. C has much stronger rules, not on casting, but on accessing the >> pointees, which basically invalidates your argument. Refer to the C >> standard for details. > > Really? What rules? Look at the C11 standard, section 6.3.2.3 ("Pointers"), 6.5.?6-7 ("effective types"), and 6.5.3.2 ("Address and indirection operators"). It is tiring to constantly correct misunderstandings about pointer casting and dereferencing. > $ cat demo.c; gcc -Wall demo.c; ./a.out [...] If you don't know what undefined behavior is, better avoid C. Your program has UB, anything can happen, including a seemingly sensible result. >> But you can modify the class (not __class__) in whatever way you want. >> For instance: >> >> class X(object): >> def f(self): ... >> ... >> del X.f >> >> So, in Python, knowing that object x is an instance of class X tells >> you... essentially nothing. Call this strong typing if you want. In >> terms of type systems, it is (strong) simplistic-typing based on type >> labels, and labels carry no information whatsoever. > > Sure you can. And you can do a lot of other things at run time, too. > Monkey-patching doesn't change the fact that x really and truly is an > instance of class X, it just changes what you can do with that object. > So what you're saying is that Python's dynamism makes it possible to > disrupt duck typing. Sure, I'll grant you that. But the type system > itself isn't broken by that. I didn't say it is broken. I said: it does very little. -- Alain. From rosuav at gmail.com Mon Feb 19 05:05:28 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 19 Feb 2018 21:05:28 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <8af6bb82-fef7-96d0-279c-037568b81aa1@vub.be> References: <8af6bb82-fef7-96d0-279c-037568b81aa1@vub.be> Message-ID: On Mon, Feb 19, 2018 at 8:36 PM, Antoon Pardon wrote: > On 17-02-18 21:11, Chris Angelico wrote: >> On Sun, Feb 18, 2018 at 1:47 AM, Ian Kelly wrote: >>> On Fri, Feb 16, 2018 at 9:32 PM, Chris Angelico wrote: >>>> You'd be surprised how rarely that kind of performance even matters. >>>> The author of that article cites C# as a superior language, but in the >>>> rewrite from C# to Python (the same one I mentioned in the other >>>> post), I sped the program up incredibly. Part of that is because C# >>>> requires the startup of its framework (in my case that's Mono) just as >>>> Python does, and partly it's because the simplicity of Python let me >>>> eliminate a number of unnecessary HTTP requests. Trust me, your choice >>>> of language doesn't help you if it means you do three (sequential) >>>> HTTP requests when one would have done. Clean code has its own >>>> advantages. >>> Okay, I'm curious. How did C# force you to make extra HTTP requests >>> that were no longer necessary when you rewrote in Python? >> It didn't *force* those requests to be made, but the code was so large >> and convoluted that I doubt its original author realized that the >> requests were being repeated. Pseudo-coded: > > But was that a reflection on C# or on the original author? > A bit of both. It's the language's fault that simple operations take dozens of lines of code, and the author's fault for not noticing the redundancies. It's virtually impossible to keep thousands of lines of code in your head, so the language's lack of good primitives increases the complexity of the code and thus allows redundancy to lie hidden. But my main point in the original post was that C# cannot possibly *help* if most of the time is spent in HTTP requests. I could have naively transformed the code into Python without seeing any measurable performance drop, despite Python being "slower" than C#. Makes no difference what language you write in, if you're waiting on the network... ChrisA From rosuav at gmail.com Mon Feb 19 05:09:25 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 19 Feb 2018 21:09:25 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <87a7w5uwg0.fsf@universite-de-strasbourg.fr.invalid> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> <87a7w5uwg0.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Mon, Feb 19, 2018 at 9:04 PM, Alain Ketterlin wrote: > Chris Angelico writes: > >> On Mon, Feb 19, 2018 at 7:40 PM, Alain Ketterlin >> wrote: > >>> No. C has much stronger rules, not on casting, but on accessing the >>> pointees, which basically invalidates your argument. Refer to the C >>> standard for details. >> >> Really? What rules? > > Look at the C11 standard, section 6.3.2.3 ("Pointers"), 6.5.?6-7 > ("effective types"), and 6.5.3.2 ("Address and indirection operators"). > It is tiring to constantly correct misunderstandings about pointer > casting and dereferencing. > >> $ cat demo.c; gcc -Wall demo.c; ./a.out > [...] > > If you don't know what undefined behavior is, better avoid C. Your > program has UB, anything can happen, including a seemingly sensible > result. Sure it can. But I don't know what you can mean by "stronger rules" if all it says is "that's undefined". Saying that behaviour is undefined does NOT mean that C has a stronger type system. I got no errors, not even a warning in -Wall mode, so there is no indication that my code did something wrong. From steve+comp.lang.python at pearwood.info Mon Feb 19 05:24:20 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 10:24:20 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Mon, 19 Feb 2018 09:40:09 +0100, Alain Ketterlin wrote: > Tim Delaney writes: > > [...] >> As others have said, typing is about how the underlying memory is >> treated. > > No. It is much more than that. Typing is about everything you can say > about a given statement. "Everything"? Truly *everything*? Given: # engage the type system somehow... # declare a, b, c: non-negative integers a = abs(int(input("Give me an integer"))) b = a*a c = (a+1)*(a+1) can you give me an example of a type-system which is capable of telling me that: if (c - b) % 2 == 1: print("hello world") else: fire_missiles() will never fire the missiles? I'd be impressed enough with a type system that knew that a%2 was always 0 or 1, although I suppose there could be some that already know that. Hell, I'd even be impressed if it could tell that c was not zero... -- Steve From alain at universite-de-strasbourg.fr.invalid Mon Feb 19 05:43:11 2018 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Mon, 19 Feb 2018 11:43:11 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> <87a7w5uwg0.fsf@universite-de-strasbourg.fr.invalid> Message-ID: <87606tuuo0.fsf@universite-de-strasbourg.fr.invalid> Chris Angelico writes: > On Mon, Feb 19, 2018 at 9:04 PM, Alain Ketterlin > wrote: >> Look at the C11 standard, section 6.3.2.3 ("Pointers"), 6.5.?6-7 >> ("effective types"), and 6.5.3.2 ("Address and indirection operators"). >> It is tiring to constantly correct misunderstandings about pointer >> casting and dereferencing. >> >>> $ cat demo.c; gcc -Wall demo.c; ./a.out >> [...] >> >> If you don't know what undefined behavior is, better avoid C. Your >> program has UB, anything can happen, including a seemingly sensible >> result. > > Sure it can. But I don't know what you can mean by "stronger rules" if > all it says is "that's undefined". Saying that behaviour is undefined > does NOT mean that C has a stronger type system. I got no errors, not > even a warning in -Wall mode, so there is no indication that my code > did something wrong. I used "stronger rules" in response to the OP claim ("C lets you manipulate memory freely"). Undefined behavior is part of the semantics of C, the same way infinite loops of signed-integer overflow are. What compilers can do about it is a different matter. This is well documented, those that want to ignore UB are on their own. Anyway, I think it is irrelevant here. Search for "What every programmer should know about undefined behavior" if you are interested. -- Alain. From rosuav at gmail.com Mon Feb 19 06:15:41 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 19 Feb 2018 22:15:41 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Mon, Feb 19, 2018 at 9:24 PM, Steven D'Aprano wrote: > On Mon, 19 Feb 2018 09:40:09 +0100, Alain Ketterlin wrote: > >> Tim Delaney writes: >> >> [...] >>> As others have said, typing is about how the underlying memory is >>> treated. >> >> No. It is much more than that. Typing is about everything you can say >> about a given statement. > > "Everything"? Truly *everything*? > > Given: > > # engage the type system somehow... > # declare a, b, c: non-negative integers > a = abs(int(input("Give me an integer"))) > b = a*a > c = (a+1)*(a+1) > > can you give me an example of a type-system which is capable of telling > me that: > > if (c - b) % 2 == 1: > print("hello world") > else: > fire_missiles() > > will never fire the missiles? > > I'd be impressed enough with a type system that knew that a%2 was always > 0 or 1, although I suppose there could be some that already know that. > > Hell, I'd even be impressed if it could tell that c was not zero... > A type system, per se? I don't think so. But static analysis CAN figure out that sort of thing (obviously, since we as humans can do it), and I wouldn't be surprised in the least if tools like Coverity could detect this. ChrisA From alain at universite-de-strasbourg.fr.invalid Mon Feb 19 06:19:14 2018 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Mon, 19 Feb 2018 12:19:14 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: <871shhuszx.fsf@universite-de-strasbourg.fr.invalid> Steven D'Aprano writes: > On Mon, 19 Feb 2018 09:40:09 +0100, Alain Ketterlin wrote: > >> Tim Delaney writes: >> >> [...] >>> As others have said, typing is about how the underlying memory is >>> treated. >> >> No. It is much more than that. Typing is about everything you can say >> about a given statement. > > "Everything"? Truly *everything*? Everything you can say. > Given: > > # engage the type system somehow... > # declare a, b, c: non-negative integers > a = abs(int(input("Give me an integer"))) > b = a*a > c = (a+1)*(a+1) > > can you give me an example of a type-system which is capable of telling > me that: > > if (c - b) % 2 == 1: > print("hello world") > else: > fire_missiles() > > will never fire the missiles? Your example is ridiculously simple for all theorem provers I know (not on Python code of course, since you can't even be sure int() etc. have not been redefined). Here is one that makes your point much better: if a**n + b**n == c**n: print("hello world") else: fire_missiles() > I'd be impressed enough with a type system that knew that a%2 was always > 0 or 1, although I suppose there could be some that already know that. > > Hell, I'd even be impressed if it could tell that c was not zero... Your claim essentially is: since we cannot prove everything, let's not even try to prove anything. Go on if you think this is the right way to think about typing. -- Alain. From renting at astron.nl Mon Feb 19 06:39:51 2018 From: renting at astron.nl (Adriaan Renting) Date: Mon, 19 Feb 2018 12:39:51 +0100 Subject: [OT] multicore/cpu history Re: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: <5A8AB7870200001B0001DF67@smtp1.astron.nl> Adriaan Renting | Email: renting at astron.nl Software Engineer Radio Observatory ASTRON | Phone: +31 521 595 100 (797 direct) P.O. Box 2 | GSM: +31 6 24 25 17 28 NL-7990 AA Dwingeloo | FAX: +31 521 595 101 The Netherlands | Web: http://www.astron.nl/~renting/ >>> On 17-2-2018 at 22:02, in message , Chris Angelico wrote: > On Sun, Feb 18, 2018 at 5:05 AM, Steven D'Aprano > wrote: >> On Sat, 17 Feb 2018 15:25:15 +1100, Chris Angelico wrote: >> ... >>> Totally not true. The GIL does not stop other threads from running. >>> Also, Python has existed for multiple CPU systems pretty much since its >>> inception, I believe. (Summoning the D'Aprano for history lesson?) >> >> If you're talking about common desktop computers, I think you're >> forgetting how recent multicore machines actually are. I'm having >> difficulty finding when multicore machines first hit the market, but it >> seems to have been well into the 21st century -- perhaps as late as 2006 >> with the AMD Athelon 64 X2: > > No, I'm talking about big iron. Has Python been running on multi-CPU > supercomputers earlier than that? > >> By the way, multiple CPU machines are different from CPUs with multiple >> cores: >> >> http://smallbusiness.chron.com/multiple-cpu-vs-multicore-33195.html > > Yeah, it was always "multiple CPUs", not "multiple cores" when I was > growing up. And it was only ever in reference to the expensive > hardware that I could never even dream of working with. I was always > on the single-CPU home-grade systems. > Multicore became a thing with the Pentium 4 hyperthreading around ~2002 for consumers, and multi cpu was a thing much longer, even with "consumer grade" hardware: I remember running 2 Mendocino 300 MHz Celerons on a Pentium II Xeon motherboard to get a multi-cpu machine for running multiple virtual machines for testing purposes around 1998. This was not as Intel intended, but a quite cheap consumer grade hardware solution. ... > > ChrisA From rosuav at gmail.com Mon Feb 19 06:44:27 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 19 Feb 2018 22:44:27 +1100 Subject: [OT] multicore/cpu history Re: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <5A8AB7870200001B0001DF67@smtp1.astron.nl> References: <5A8AB7870200001B0001DF67@smtp1.astron.nl> Message-ID: On Mon, Feb 19, 2018 at 10:39 PM, Adriaan Renting wrote: > I remember running 2 Mendocino 300 MHz Celerons on a Pentium II Xeon > motherboard to get a > multi-cpu machine for running multiple virtual machines for testing > purposes around 1998. > This was not as Intel intended, but a quite cheap consumer grade > hardware solution. > Thanks! That's the sort of thing I was looking for. Out of curiosity, what was the purpose of that rig and its dual CPUs? Were you running the host system on one CPU and the guest on another? ChrisA From marko at pacujo.net Mon Feb 19 07:23:43 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 19 Feb 2018 14:23:43 +0200 Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> <871shhuszx.fsf@universite-de-strasbourg.fr.invalid> Message-ID: <87k1v9mals.fsf@elektro.pacujo.net> Alain Ketterlin : > Your claim essentially is: since we cannot prove everything, let's not > even try to prove anything. Go on if you think this is the right way to > think about typing. This discussion is far too metaphysical. Static type declarations give you something at a cost. They give you: * Performance (by several orders of magnitude). * Static type checking (-> better quality). They cost: * More code to type (-> worse quality, lower productivity). In my experience it is far easier to produce correct code in Python than in, say, C++ or Java. Back to metaphysics: OO has spent far too much energy in ontology. You shouldn't judge an object based on the class it belongs to. Marko From rhodri at kynesim.co.uk Mon Feb 19 07:32:49 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 19 Feb 2018 12:32:49 +0000 Subject: Python 2 to 3 Conversion In-Reply-To: References: Message-ID: On 18/02/18 16:18, Wildman via Python-list wrote: >> But that's only going to show one (uplink) address. If I needed to get >> ALL addresses for ALL network adapters, I'd either look for a library, >> and if one wasn't easily found, I'd shell out to the "ip" command and >> parse its output.:) >> > I considered using the "ip" command but I prefer not to > depend on external programs if I can get around it. I > know that might be considered silly but that's just me. It's not silly at all, but for Linux networking it might be the best idea. I believe in theory you are supposed to use libnl (in some suitable wrapping), but my experience with that is that it is badly documented and horrendously unreliable. -- Rhodri James *-* Kynesim Ltd From bc at freeuk.com Mon Feb 19 07:35:19 2018 From: bc at freeuk.com (bartc) Date: Mon, 19 Feb 2018 12:35:19 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> Message-ID: <7wziC.125009$BM3.115116@fx14.am4> On 19/02/2018 02:59, Chris Angelico wrote: > On Mon, Feb 19, 2018 at 1:14 PM, bartc wrote: >> How would even a type for the odd numbers from 1 to 10 inclusive work? >> (That, a type consisting of one of the values in {1,3,5,7,9}.) Would they be >> ordered or unordered? Can I do arithmetic with them: will 3*3 work, but not >> 3*5? > > The type is "positive odd number below ten" and could be written as > int(1..9|1%2). That is an orderable type; you can say that 3 < 7, for > instance. And yes, arithmetic would be defined just fine; Sometimes, the reason for creating a special numerical type is precisely so you can't do arithmetic on them, if it's not meaningful for the type. So the special type of the values 65..90 might not allow the type be multiplied or divided, or added to itself. Because they represent characters A..Z. Or house numbers. Or the age of pensioners. (You'd need to convert to ordinary integers, is that is allowed.) there's no > requirement for the result of an operation to have the same type as > its inputs: > >>>> 5 / 2 # two integers > 2.5 Try that when the type of {1..13} represents playing card ordinal values. Type systems get rapidly very complicated when you have to deal with arbitrary sets of values and with arbitrary rules of interaction. Someone has to devise a programming language to allow all that without tying itself up in knots. Someone else has to program in it. And someone else has to try and understand it! Ones like C++ has already tied itself itself up in knots just doing the basics; I'm not sure how it would handle even my 1,3,5,7,9 type. But Python has classes and can do some of this stuff; how would it handle a numeric type that is constrained to be whole numbers within 0..9 inclusive? -- bartc From fabiofz at gmail.com Mon Feb 19 07:48:03 2018 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Mon, 19 Feb 2018 09:48:03 -0300 Subject: Python with PyDev on Visual Studio Code Message-ID: Hi All, I'm happy to announce that PyDev (http://www.pydev.org) can now be used for Python development on Visual Studio Code! The first release already provides features such as code analysis, code completion, go to definition, symbols for the workspace and editor, code formatting, find references, quick fixes and more (see http://www.pydev.org/vscode/ for details). All features have a strong focus on speed and have been shaped by the usage on PyDev over the last 14 years, so, I believe it's already pretty nice to use... there are still some big things to integrate (such as the PyDev debugger), but those should come on shortly. The requisites are having java 8 (or higher) installed on the system (if it doesn't find it automatically the java home location may need to be specified in the settings -- http://www.pydev.org/vscode/ has more details) and Python 2.6 or newer. By default it should pick the python executable available on the PATH, but it's possible to specify a different python executable through the settings on VSCode (see http://www.pydev.org/vscode/settings.html for details). Below, I want to share some of the things that are unique in PyDev and are now available for VSCode users: - Niceties from PyDev when typing such as auto-adding self where needed (note that having the editor.formatOnType setting turned on is a requisite for that to work). - Really fast code-completion, code-analysis and code-formatting engines. - Code completion provides options to import modules, top level classes, methods and variables (python.pydev.preferredImportLocation can be used to determine the location of the import). - Quick fix which automatically allows adding an import for unresolved symbols. - In-file navigation to previous or next class or method through Ctrl+Shift+Up and Ctrl+Shift+Down. See: http://www.pydev.org/vscode/ for more information! Cheers, -- Fabio Zadrozny ------------------------------ Software Developer PyDev on VSCode http://pydev.org/vscode PyVmMonitor - Profile Python on VSCode http://www.pyvmmonitor.com/ From wegge at wegge.dk Mon Feb 19 08:06:36 2018 From: wegge at wegge.dk (Anders Wegge Keller) Date: Mon, 19 Feb 2018 14:06:36 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> Message-ID: <20180219140636.1dd40136@wegge.dk> P? Mon, 19 Feb 2018 04:39:31 +0000 (UTC) Steven D'Aprano skrev: > On Mon, 19 Feb 2018 04:26:32 +0100, Anders Wegge Keller wrote: > > > P? Mon, 19 Feb 2018 08:47:14 +1100 > > Tim Delaney skrev: > >> On 18 February 2018 at 22:55, Anders Wegge Keller > >> wrote: > > > > > [...] > > > >> You couldn't have got the above much more wrong. > > > >> As others have said, typing is about how the underlying memory is > >> treated. > > > > And that is exactly my point. Python does not have a typed list. It > > have a list that takes whatever is thrown into it. > > > > I'll skip the rest, as you totally missed the point. > > I think its actually you have totally missed the point. What you want is > a homogeneous list, a list which only accepts items of the same type. The > built-in list is not that data structure, but Python already has > something similar: see the array module. Given that I'm the one making a point about the unwarranted smugness, I know that I'm not missing anything. Array is not even close to providing a strongly typed container. For all of the yakking about other languages weak, blah, BCPL, blah, I still wonder where that need to feel superior to anything else comes from. Python isn't particular strong typed. In fact, apart from asking an object what type it is, types are not that important. It's the interface that matters. I wonder why this is a sore point for Python developers? -- //Wegge From elchino at cnn.cn Mon Feb 19 08:08:37 2018 From: elchino at cnn.cn (ElChino) Date: Mon, 19 Feb 2018 14:08:37 +0100 Subject: Python with PyDev on Visual Studio Code In-Reply-To: References: Message-ID: Fabio Zadrozny wrote: > See: http://www.pydev.org/vscode/ for more information! That page includes so many dead links that it looks like a joke. From p.f.moore at gmail.com Mon Feb 19 08:28:26 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 19 Feb 2018 13:28:26 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <20180219140636.1dd40136@wegge.dk> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: On 19 February 2018 at 13:06, Anders Wegge Keller wrote: > Python isn't particular strong typed. In fact, apart from asking an object > what type it is, types are not that important. It's the interface that > matters. I wonder why this is a sore point for Python developers? Because there's a long history of people claiming that "strongly typed" languages are fundamentally better than "scripting languages" (and putting Python in the "scripting language" class) without either being clear about what "strongly typed" means, or about whether Python is actually strongly typed or not[1], maybe? The reality is that the term "strongly typed" can be made to mean whatever you want it to mean in these debates, and such claims usually turn out to be little more than statements "yah boo my language is better than yours and your language sucks". Paul [1] The most basic question, which people making such claims often can't answer, is "Do you mean that values are strongly typed, or that names are? Or did you mean that variables are, because if so Python doesn't even have variables in the sense that you mean" Programming language semantics are complex. From steve+comp.lang.python at pearwood.info Mon Feb 19 08:34:58 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 13:34:58 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Mon, 19 Feb 2018 20:14:32 +1100, Chris Angelico wrote: > As an integer, 3.141590 is 1078530000 $ > > Looks to me like C is perfectly happy to interpret a float as an int. Yes, but that's not an *automatic* coercion. To count as weakly typed, the compiler has to do it automatically, without an explicit cast or conversion. I understand well that many people criticise C for making it too easy for the programmer to violate type-safety (or even for allowing it *at all*), but that's an orthogonal issue -- and probably essential for a systems language. D, Rust and other new-generation systems languages which have much stronger type systems than C nevertheless also allow programmers to escape from the type system when necessary. -- Steve From fabiofz at gmail.com Mon Feb 19 08:48:07 2018 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Mon, 19 Feb 2018 10:48:07 -0300 Subject: Python with PyDev on Visual Studio Code In-Reply-To: References: Message-ID: Sorry, it was a glitch on the template when moved to another folder (just fixed). On Mon, Feb 19, 2018 at 10:08 AM, ElChino wrote: > Fabio Zadrozny wrote: > > See: http://www.pydev.org/vscode/ for more information! >> > > That page includes so many dead links that it looks > like a joke. > -- > https://mail.python.org/mailman/listinfo/python-list > From steve+comp.lang.python at pearwood.info Mon Feb 19 09:16:57 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 14:16:57 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> <871shhuszx.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Mon, 19 Feb 2018 12:19:14 +0100, Alain Ketterlin wrote: > Steven D'Aprano writes: > >> On Mon, 19 Feb 2018 09:40:09 +0100, Alain Ketterlin wrote: >> >>> Tim Delaney writes: >>> >>> [...] >>>> As others have said, typing is about how the underlying memory is >>>> treated. >>> >>> No. It is much more than that. Typing is about everything you can say >>> about a given statement. >> >> "Everything"? Truly *everything*? > > Everything you can say. > >> Given: >> >> # engage the type system somehow... >> # declare a, b, c: non-negative integers >> a = abs(int(input("Give me an integer"))) >> b = a*a >> c = (a+1)*(a+1) >> >> can you give me an example of a type-system which is capable of telling >> me that: >> >> if (c - b) % 2 == 1: >> print("hello world") >> else: >> fire_missiles() >> >> will never fire the missiles? > > Your example is ridiculously simple for all theorem provers I know (not > on Python code of course, since you can't even be sure int() etc. have > not been redefined). I didn't ask about theorem provers. I asked about type systems. https://en.wikipedia.org/wiki/Automated_theorem_prover One could, in principle, add a theorem prover to a type system, or at least add certain theorem-proving-like functionality to it, but at what point does the type checker become complex enough that we can no longer trust it? If you don't understand the proof that code is bug-free, you're really trusting that the type-checker is bug-free. > Here is one that makes your point much better: > > if a**n + b**n == c**n: > print("hello world") > else: > fire_missiles() Either way, I doubt any existing type systems could eliminate the dead code in those examples. If you know of one which would, I'd be interested to hear about it. >> I'd be impressed enough with a type system that knew that a%2 was >> always 0 or 1, although I suppose there could be some that already know >> that. >> >> Hell, I'd even be impressed if it could tell that c was not zero... > > Your claim essentially is: since we cannot prove everything, let's not > even try to prove anything. Go on if you think this is the right way to > think about typing. That's not what I am saying. You're putting words into my mouth and criticising me for a position much more extreme than I've expressed. If you search the archives to this list, you'll see that I've frequently supported Python adding type-hints to the language[1], to allow tools like MyPy and various linters to perform static analysis on Python code. Especially for large projects, that can be helpful. Type-checking is yet another tool for gathering evidence (not "proving") that a program is correct, together with (among others) code review and especially testing. As Donald Knuth said: "Beware of bugs in the above code; I have only proved it correct, not tried it." What I object to is the idea that static type-checking is "indispensable" or sufficient. Its one tool out of many, and more dispensable than some of the others. Even in a big project, which would you rather give up: all testing, or all static type checking? I don't just mean all formal tests, but even running the code to see if it works. Literally "it compiles, ship it!". No, I don't think so. Testing is truly indispensable, whether they are formal tests or not. Static typing and "type-safety", not so much. [1] If you search far back enough, you may find me taking a more extreme position that static typing was unnecessary. That was a much younger and more naive me. -- Steve From steve+comp.lang.python at pearwood.info Mon Feb 19 09:25:26 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 14:25:26 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: On Mon, 19 Feb 2018 12:35:19 +0000, bartc wrote: > Type systems get rapidly very complicated when you have to deal with > arbitrary sets of values and with arbitrary rules of interaction. > Someone has to devise a programming language to allow all that without > tying itself up in knots. Someone else has to program in it. And someone > else has to try and understand it! Indeed. That's exactly the point I'm making. A type-system that allowed you to express extremely fine-grained distinctions would be effectively a fully fledged programming language itself, probably even Turing Complete, and so our type-specifications would be as error-prone as the code we write now. Probably more so, due to the requirements that type specifications be a lot more compact than the rest of the code we write. > Ones like C++ has already tied itself itself up in knots just doing the > basics; I'm not sure how it would handle even my 1,3,5,7,9 type. > > But Python has classes and can do some of this stuff; how would it > handle a numeric type that is constrained to be whole numbers within > 0..9 inclusive? This becomes easy at run-time: class Digit(int): def __new__(cls, arg): instance = super().__new__(cls, arg) if not 0 <= instance <= 9: raise ValueError('argument is out of range') return instance The above is obviously not a full-blown production-ready class. But it illustrates the basic concept. This is the sort of thing that dynamic languages excel at: enforcing constraints at run-time which are hard to enforce at compile-time. -- Steve From steve+comp.lang.python at pearwood.info Mon Feb 19 09:54:55 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 14:54:55 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: On Mon, 19 Feb 2018 13:28:26 +0000, Paul Moore wrote: > [1] The most basic question, which people making such claims often can't > answer, is "Do you mean that values are strongly typed, or that names > are? Or did you mean that variables are, because if so Python doesn't > even have variables in the sense that you mean" Programming language > semantics are complex. An excellent point. The distinction between typing *values* and *names* is a good one. -- Steve From steve+comp.lang.python at pearwood.info Mon Feb 19 10:15:19 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 19 Feb 2018 15:15:19 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: On Mon, 19 Feb 2018 14:06:36 +0100, Anders Wegge Keller wrote: > Array is not even close to providing a strongly typed container. That's a mighty powerful claim that goes against the documentation for the array module. Can you back your claims up? Here's an array and a list: import array arr = array.array('I') lst = [] Without cheating (more on this below), your challenge is to do *any* of the following (your choice): 1. Change the type of the object currently assigned to arr to a string (or any other non-array type of your choice); OR 2. Change the type of the object currently assigned to lst to a string (or any other non-list type of your choice); OR 3. Assign a string (or any other non-integer value of your choice) to any position in the array; OR 4. Assign a negative integer to any position in the array. If you can do any of those things (with the restrictions given below), I'll happily acknowledge that I was wrong, you were right, and Python's "strong typing" is much weaker than I thought (or even non-existent). On the other hand, if you can't do any of them, I expect you to acknowledge that you were wrong. Here are the restrictions ("no cheating"): - You can't replace, shadow, monkey-patch or otherwise modify the array module or the array.array type. - Assigning a new object to the variable name "arr" does not count as changing the type of the existing array object. You must change the type of the instance, not replace it with another instance. - Likewise for the lst variable name. You must change the type of the object, not re-assign a new object to the same name. - Any sort of hack involving ctypes is interesting, but doesn't count; we know that ctypes can break all sorts of invariants by manipulating the C engine. Only standard Python code is permitted. - Likewise you aren't allowed to patch the interpreter. It must be the standard CPython interpreter, version 2.4 or greater. - Nor are you allowed to "cheat" by redefining builtins like print, repr, type, id, sys.displayhook, sys.stdout etc in order to fake output. - Exploiting bugs in the interpreter to cause side-effects don't count (its a bug, not a language feature). - Any other clever trick that goes against the spirit of the requirements will be interesting to see, but not count as a success. The spirit of the requirements are that you demonstrate the ability to do something like this: old_lst = lst # get a second reference to the list object assert type(lst) is type([]) do_stuff(lst) # do something clever to the list object... assert type(lst) is not type([]) # OMG you changed the type! assert old_lst is lst # still the same object (Similar for arr.) I don't believe that you can do this. I believe that you have failed to understand what we're talking about when we say Python has strong, dynamic typing, but I'll be happy for you to prove me wrong. -- Steve From ned at nedbatchelder.com Mon Feb 19 10:18:37 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 19 Feb 2018 10:18:37 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: On 2/19/18 9:54 AM, Steven D'Aprano wrote: > On Mon, 19 Feb 2018 13:28:26 +0000, Paul Moore wrote: > >> [1] The most basic question, which people making such claims often can't >> answer, is "Do you mean that values are strongly typed, or that names >> are? Or did you mean that variables are, because if so Python doesn't >> even have variables in the sense that you mean" Programming language >> semantics are complex. > An excellent point. > > The distinction between typing *values* and *names* is a good one. > > > I guess I'll have to continue to grit my teeth as people say, "Python doesn't have variables."?? Why can't we say, "Python's variables work differently than other languages"? --Ned. From rosuav at gmail.com Mon Feb 19 10:26:19 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 02:26:19 +1100 Subject: Python 2 to 3 Conversion In-Reply-To: References: Message-ID: On Mon, Feb 19, 2018 at 11:32 PM, Rhodri James wrote: > On 18/02/18 16:18, Wildman via Python-list wrote: >>> >>> But that's only going to show one (uplink) address. If I needed to get >>> ALL addresses for ALL network adapters, I'd either look for a library, >>> and if one wasn't easily found, I'd shell out to the "ip" command and >>> parse its output.:) >>> >> I considered using the "ip" command but I prefer not to >> depend on external programs if I can get around it. I >> know that might be considered silly but that's just me. > > > It's not silly at all, but for Linux networking it might be the best idea. > I believe in theory you are supposed to use libnl (in some suitable > wrapping), but my experience with that is that it is badly documented and > horrendously unreliable. Agreed. Obviously the first choice is to look in the standard library for the functionality you want, but it's not there (at least, I couldn't find it). Then your next choices, if I'm not mistaken, are: * Random third-party packages you might find on PyPI * Opaque IOCTLs * Wrapping a poorly-documented C library * Invoking a subprocess and parsing its output When those are the choices, the subprocess option doesn't look too bad. ChrisA From marko at pacujo.net Mon Feb 19 10:28:18 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 19 Feb 2018 17:28:18 +0200 Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: <87a7w5m225.fsf@elektro.pacujo.net> Ned Batchelder : > I guess I'll have to continue to grit my teeth as people say, "Python > doesn't have variables."?? Why can't we say, "Python's variables work > differently than other languages"? Because they don't? Marko From wegge at wegge.dk Mon Feb 19 10:34:29 2018 From: wegge at wegge.dk (Anders Wegge Keller) Date: Mon, 19 Feb 2018 16:34:29 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: <20180219163429.048ba054@wegge.dk> P? Mon, 19 Feb 2018 15:15:19 +0000 (UTC) Steven D'Aprano skrev: > On Mon, 19 Feb 2018 14:06:36 +0100, Anders Wegge Keller wrote: > > > Array is not even close to providing a strongly typed container. > > That's a mighty powerful claim that goes against the documentation for > the array module. Can you back your claims up? > > Here's an array and a list: Make me an array of tuples with two integers and a string, and we can talk. -- //Wegge From rosuav at gmail.com Mon Feb 19 10:35:37 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 02:35:37 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Tue, Feb 20, 2018 at 12:34 AM, Steven D'Aprano wrote: > On Mon, 19 Feb 2018 20:14:32 +1100, Chris Angelico wrote: > >> As an integer, 3.141590 is 1078530000 $ >> >> Looks to me like C is perfectly happy to interpret a float as an int. > > Yes, but that's not an *automatic* coercion. To count as weakly typed, > the compiler has to do it automatically, without an explicit cast or > conversion. Fair enough. If you ignore warnings, then C does have that kind of weak typing: $ cat demo.c #include int main() { float f = 3.14159; int *i = &f; printf("As an integer, %f is %d\n", f, *i); return 0; } $ gcc demo.c demo.c: In function ?main?: demo.c:5:14: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types] int *i = &f; ^ $ GCC was quite happy to compile that code, even though the type of "&f" is "pointer to float", and it's being assigned to a variable of type "pointer to int". But C is a language saddled with so much history and backward compatibility constraints that there are some things you just CAN'T make into errors; so in terms of "how safe is C?", I'd have to say that warnings (especially those that are enabled by default - I didn't need to say "-Wall" for this test), count as "disallowing", especially if all the major compilers emit warnings on the same code. But I do see the argument that "it compiles, so the language clearly permits it". ChrisA From p.f.moore at gmail.com Mon Feb 19 10:39:00 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 19 Feb 2018 15:39:00 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: On 19 February 2018 at 15:18, Ned Batchelder wrote: > On 2/19/18 9:54 AM, Steven D'Aprano wrote: >> >> On Mon, 19 Feb 2018 13:28:26 +0000, Paul Moore wrote: >> >>> [1] The most basic question, which people making such claims often can't >>> answer, is "Do you mean that values are strongly typed, or that names >>> are? Or did you mean that variables are, because if so Python doesn't >>> even have variables in the sense that you mean" Programming language >>> semantics are complex. >> >> An excellent point. >> >> The distinction between typing *values* and *names* is a good one. >> > > I guess I'll have to continue to grit my teeth as people say, "Python > doesn't have variables." Why can't we say, "Python's variables work > differently than other languages"? Apologies, Ned. I didn't mean to make your teeth sore :-) I've found that when people refer to "variables" they pretty much inevitably have a picture in their mind of a named box that "contains" a value (or pointer to a value). They find it difficult to understand that Python doesn't have those boxes but rather names the values directly. Sort of. And yes, that 's very much "sort of". It's certainly just as viable to say to people that Python's idea of variables is different than (say) C's, but in my experience you end up qualifying so much that it's easier to use a different term without the connotations. Sort of like saying "I like apples - the orange ones that are a type of citrus fruit" :-) In common usage I'll happily use the terms "variable" and "name" somewhat interchangeably, but when things degenerate into detail-level picking over specifics, I avoid doing so. I was wrong to casually say "Python doesn't have variables", though - if I want to be detailed and specific, I should make sure my writing reflects that that's what I'm doing. I'm curious - How would you explain Python's "variables" to someone who knows how C variables work, in a way that ensures they don't carry across any unfortunate misconceptions based on how C works? If I had a good way of doing that, maybe I wouldn't need to play apple/orange games when discussing the subject. Paul From rosuav at gmail.com Mon Feb 19 10:45:57 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 02:45:57 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <7wziC.125009$BM3.115116@fx14.am4> References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: On Mon, Feb 19, 2018 at 11:35 PM, bartc wrote: > Sometimes, the reason for creating a special numerical type is precisely so > you can't do arithmetic on them, if it's not meaningful for the type. > > So the special type of the values 65..90 might not allow the type be > multiplied or divided, or added to itself. Because they represent characters > A..Z. Or house numbers. Or the age of pensioners. (You'd need to convert to > ordinary integers, is that is allowed.) If they represent characters, then they're not integers. They're characters. Characters are not integers. House numbers and pensioners' ages are not unitless numbers. If you truly want to represent these as actual types, then you need to incorporate the unit in the type, or have a generic "number and unit" data type. That would then change the operations permitted on that type. None of this says anything about the original question, which was about numbers. Actual, plain, ordinary integers. >> there's no >> requirement for the result of an operation to have the same type as >> its inputs: >> >>>>> 5 / 2 # two integers >> >> 2.5 > > Try that when the type of {1..13} represents playing card ordinal values. And now you're pretending that an enumeration is the same thing as an integer. You're arguing about a type system, then trying to claim that everything that can be represented as an integer must be an integer. Your position is illogical. > But Python has classes and can do some of this stuff; how would it handle a > numeric type that is constrained to be whole numbers within 0..9 inclusive? Do you mean that arithmetic results have to also be constrained, or that the numeric type must itself only be those few? You could fairly easily define a data type like this: Int[0:10] # integer 0...9 inclusive (0..10 inc/exc) Int[0:10] + Int[0:10] ==> Int[0:19] Int[0:10] * Int[0:10] ==> Int[0:81] Int[0:10] / Int[1:10] ==> Fraction Perfectly sane and useful. ChrisA From marko at pacujo.net Mon Feb 19 11:20:03 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 19 Feb 2018 18:20:03 +0200 Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: <87606tlznw.fsf@elektro.pacujo.net> Paul Moore : > I'm curious - How would you explain Python's "variables" to someone > who knows how C variables work, in a way that ensures they don't carry > across any unfortunate misconceptions based on how C works? Just say that 1. Every Python variable is of the type "Object *". 2. Every Python expression evaluates to a pointer (Object *). 3. Python's "." is equivalent to C's "->". That'll get the main point across pretty fast. Marko From best_lay at yahoo.com Mon Feb 19 11:49:16 2018 From: best_lay at yahoo.com (Wildman) Date: Mon, 19 Feb 2018 10:49:16 -0600 Subject: Python 2 to 3 Conversion References: Message-ID: On Mon, 19 Feb 2018 12:32:49 +0000, Rhodri James wrote: > On 18/02/18 16:18, Wildman via Python-list wrote: >>> But that's only going to show one (uplink) address. If I needed to get >>> ALL addresses for ALL network adapters, I'd either look for a library, >>> and if one wasn't easily found, I'd shell out to the "ip" command and >>> parse its output.:) >>> >> I considered using the "ip" command but I prefer not to >> depend on external programs if I can get around it. I >> know that might be considered silly but that's just me. > > It's not silly at all, but for Linux networking it might be the best > idea. I believe in theory you are supposed to use libnl (in some > suitable wrapping), but my experience with that is that it is badly > documented and horrendously unreliable. It looks like libnl would do what I want but there is a problem. When finished, my program will be released in the form of a Debian (*.deb) package and used by, for the most part, 'average' Linux users. These are people that know their way around Linux but know nothing or very little about Python. Installing a package using pip by them is pretty much out of the question. The system's package manager must be able to handle the program's dependencies so third-party packages are out of the question. But thanks for the suggestion. -- GNU/Linux user #557453 "There are only 10 types of people in the world... those who understand Binary... and those who don't." -Spike From best_lay at yahoo.com Mon Feb 19 11:53:22 2018 From: best_lay at yahoo.com (Wildman) Date: Mon, 19 Feb 2018 10:53:22 -0600 Subject: Python 2 to 3 Conversion References: Message-ID: On Tue, 20 Feb 2018 02:26:19 +1100, Chris Angelico wrote: > * Opaque IOCTLs Would you mind to elaborate a little about your concerns? -- GNU/Linux user #557453 "There are only 10 types of people in the world... those who understand Binary... and those who don't." -Spike From ned at nedbatchelder.com Mon Feb 19 12:11:36 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 19 Feb 2018 12:11:36 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: On 2/19/18 10:39 AM, Paul Moore wrote: > On 19 February 2018 at 15:18, Ned Batchelder wrote: >> On 2/19/18 9:54 AM, Steven D'Aprano wrote: >>> On Mon, 19 Feb 2018 13:28:26 +0000, Paul Moore wrote: >>> >>>> [1] The most basic question, which people making such claims often can't >>>> answer, is "Do you mean that values are strongly typed, or that names >>>> are? Or did you mean that variables are, because if so Python doesn't >>>> even have variables in the sense that you mean" Programming language >>>> semantics are complex. >>> An excellent point. >>> >>> The distinction between typing *values* and *names* is a good one. >>> >> I guess I'll have to continue to grit my teeth as people say, "Python >> doesn't have variables." Why can't we say, "Python's variables work >> differently than other languages"? > Apologies, Ned. I didn't mean to make your teeth sore :-) > > I've found that when people refer to "variables" they pretty much > inevitably have a picture in their mind of a named box that "contains" > a value (or pointer to a value). They find it difficult to understand > that Python doesn't have those boxes but rather names the values > directly. Sort of. And yes, that 's very much "sort of". It's > certainly just as viable to say to people that Python's idea of > variables is different than (say) C's, but in my experience you end up > qualifying so much that it's easier to use a different term without > the connotations. Sort of like saying "I like apples - the orange ones > that are a type of citrus fruit" :-) > > In common usage I'll happily use the terms "variable" and "name" > somewhat interchangeably, but when things degenerate into detail-level > picking over specifics, I avoid doing so. I was wrong to casually say > "Python doesn't have variables", though - if I want to be detailed and > specific, I should make sure my writing reflects that that's what I'm > doing. > > I'm curious - How would you explain Python's "variables" to someone > who knows how C variables work, in a way that ensures they don't carry > across any unfortunate misconceptions based on how C works? If I had a > good way of doing that, maybe I wouldn't need to play apple/orange > games when discussing the subject. I would (and did) explain it like this: https://nedbatchelder.com/text/names1.html That talk was pretty much powered by hating the phrase "Python has no variables" :) --Ned. > > Paul From p.f.moore at gmail.com Mon Feb 19 13:01:27 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 19 Feb 2018 18:01:27 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: On 19 February 2018 at 17:11, Ned Batchelder wrote: > On 2/19/18 10:39 AM, Paul Moore wrote: >> >> I'm curious - How would you explain Python's "variables" to someone >> who knows how C variables work, in a way that ensures they don't carry >> across any unfortunate misconceptions based on how C works? If I had a >> good way of doing that, maybe I wouldn't need to play apple/orange >> games when discussing the subject. > > I would (and did) explain it like this: > https://nedbatchelder.com/text/names1.html > > That talk was pretty much powered by hating the phrase "Python has no > variables" :) Interesting (and somewhat embarrassing :-() That talk (which I'd forgotten was yours) was one of the key things that made me start thinking in terms of Python naming values rather than assigning values to variables! I still find that your explanation (which never uses the term "variable" until you refer to the "Python has no variables" idea at the end) is one of the best ways to describe how Python assignment works. But using your explanation as a way to defend a statement that you don't agree with is wrong, so I'll stop doing that in future. Sorry! In terms of your talk, would I be right to say that "names" (in the sense you use them in that talk) are Python's "variables"? That equates to common usage, so I can go with that ("Python's variables act like names, unlike other languages"). But I'd hate to replace one misunderstanding of what you said with another, so let me know if I've still got it wrong... Paul From aewindhorn at gmail.com Mon Feb 19 13:05:05 2018 From: aewindhorn at gmail.com (windhorn) Date: Mon, 19 Feb 2018 10:05:05 -0800 (PST) Subject: How to run script from interpreter? In-Reply-To: References: Message-ID: On Saturday, February 17, 2018 at 8:50:48 AM UTC-6, Steven D'Aprano wrote: > > For me, the tool I use is a set of re-usable tools: > > - a text editor; > - a system command prompt in a terminal/console window; > - a Python REPL for running code snippets and looking up help(obj). > > Other people prefer a IDE like Spyder (or many others). Either way, > they're designed for the edit-run-debug-edit cycle. I hadn't looked at Spyder, it seems very useful. > I edit the source code of my script in the text editor, then run it from > the command prompt using > > python myscript.py > > If I need the debugger, I use: > > python -i myscript.py > ... > then edit the script to make the tests work. > > The best part of this is each time you run the script, it is guaranteed > to be running *fresh*, with no left-over cruft from previous runs. Good point. But some of that is stuff I'm using, so maybe it has to go in the script. And this from Ian seems to work for me: > py> import foo # Get the module to reload > py> reload(foo) # Reload it > py> from foo import * # Update all the names previously *-imported Thanks all for some good advice. Regards, Allen From rosuav at gmail.com Mon Feb 19 13:31:27 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 05:31:27 +1100 Subject: Python 2 to 3 Conversion In-Reply-To: References: Message-ID: On Tue, Feb 20, 2018 at 3:49 AM, Wildman via Python-list wrote: > On Mon, 19 Feb 2018 12:32:49 +0000, Rhodri James wrote: > >> On 18/02/18 16:18, Wildman via Python-list wrote: >>>> But that's only going to show one (uplink) address. If I needed to get >>>> ALL addresses for ALL network adapters, I'd either look for a library, >>>> and if one wasn't easily found, I'd shell out to the "ip" command and >>>> parse its output.:) >>>> >>> I considered using the "ip" command but I prefer not to >>> depend on external programs if I can get around it. I >>> know that might be considered silly but that's just me. >> >> It's not silly at all, but for Linux networking it might be the best >> idea. I believe in theory you are supposed to use libnl (in some >> suitable wrapping), but my experience with that is that it is badly >> documented and horrendously unreliable. > > It looks like libnl would do what I want but there is > a problem. When finished, my program will be released > in the form of a Debian (*.deb) package and used by, > for the most part, 'average' Linux users. These are > people that know their way around Linux but know > nothing or very little about Python. Installing a > package using pip by them is pretty much out of the > question. The system's package manager must be able > to handle the program's dependencies so third-party > packages are out of the question. > > But thanks for the suggestion. If you're distributing it as a .deb, you don't want to use pip to grab additional libraries. Ideally, you'd want the library to *also* be available through the package manager, which would mean you can simply record it as a dependency. If it's not... it's not gonna be easy. ChrisA From rosuav at gmail.com Mon Feb 19 13:39:15 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 05:39:15 +1100 Subject: Python 2 to 3 Conversion In-Reply-To: References: Message-ID: On Tue, Feb 20, 2018 at 3:53 AM, Wildman via Python-list wrote: > On Tue, 20 Feb 2018 02:26:19 +1100, Chris Angelico wrote: > >> * Opaque IOCTLs > > Would you mind to elaborate a little about your > concerns? Look at your original code: it's impossible to figure out what it's doing or whether it's doing it correctly. "Opaque" in this context means that you can't grok the code without external help. This is a code smell, and a strong indication that you're "fiddling" in stuff way lower level than you normally want to be working with. It's usable, but it's dirty and leaves everything up to you (do you know for sure, for instance, if this is backward compatible? Cross-platform?), and that makes it comparable to depending on an external program. ChrisA From ned at nedbatchelder.com Mon Feb 19 13:56:18 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 19 Feb 2018 13:56:18 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: On 2/19/18 1:01 PM, Paul Moore wrote: > On 19 February 2018 at 17:11, Ned Batchelder wrote: >> On 2/19/18 10:39 AM, Paul Moore wrote: >>> I'm curious - How would you explain Python's "variables" to someone >>> who knows how C variables work, in a way that ensures they don't carry >>> across any unfortunate misconceptions based on how C works? If I had a >>> good way of doing that, maybe I wouldn't need to play apple/orange >>> games when discussing the subject. >> I would (and did) explain it like this: >> https://nedbatchelder.com/text/names1.html >> >> That talk was pretty much powered by hating the phrase "Python has no >> variables" :) > Interesting (and somewhat embarrassing :-() That talk (which I'd > forgotten was yours) was one of the key things that made me start > thinking in terms of Python naming values rather than assigning values > to variables! I still find that your explanation (which never uses the > term "variable" until you refer to the "Python has no variables" idea > at the end) is one of the best ways to describe how Python assignment > works. > > But using your explanation as a way to defend a statement that you > don't agree with is wrong, so I'll stop doing that in future. Sorry! > > In terms of your talk, would I be right to say that "names" (in the > sense you use them in that talk) are Python's "variables"? That > equates to common usage, so I can go with that ("Python's variables > act like names, unlike other languages"). But I'd hate to replace one > misunderstanding of what you said with another, so let me know if I've > still got it wrong... TBH, I forget the exact words I used during the talk, but: names are Python's variables. That sounds good to me. --Ned. From stefan_ml at behnel.de Mon Feb 19 15:12:32 2018 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 19 Feb 2018 21:12:32 +0100 Subject: c code generator from python In-Reply-To: <24315cbc-86ff-4a0d-a0d0-017b658a3373@googlegroups.com> References: <24315cbc-86ff-4a0d-a0d0-017b658a3373@googlegroups.com> Message-ID: bhattacharya.kushal4 at gmail.com schrieb am 17.01.2018 um 12:03: > Is there any python framework or any tool as which can generate C code from python code as it is . http://cython.org/ Stefan From stefan_ml at behnel.de Mon Feb 19 15:23:38 2018 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 19 Feb 2018 21:23:38 +0100 Subject: Help on convert PyObject to string (c) Python 3.6 In-Reply-To: References: Message-ID: Jason Qian via Python-list schrieb am 04.02.2018 um 17:52: > This is the case of calling python from c and the python function will > return a string. Hi Jason, I noticed that you ran into a couple of problems using the C-API, so I suggest looking into Cython instead. It basically translates Python to C and supports optional static usage of C and C++ data types, so you get all the native code interaction *and* all the Python interaction and features, without having to go through the hassle of learning how the Python internals work (and why they don't work for you). The code it generates is probably also faster and safer than what you are currently writing (no offence, just experience from reading and writing a lot of such code). http://cython.org/ Stefan From jqian at tibco.com Mon Feb 19 15:58:59 2018 From: jqian at tibco.com (Jason Qian) Date: Mon, 19 Feb 2018 15:58:59 -0500 Subject: Help on convert PyObject to string (c) Python 3.6 In-Reply-To: References: Message-ID: Thanks a lot and I will take a look Cython, On Mon, Feb 19, 2018 at 3:23 PM, Stefan Behnel wrote: > Jason Qian via Python-list schrieb am 04.02.2018 um 17:52: > > This is the case of calling python from c and the python function > will > > return a string. > > Hi Jason, > > I noticed that you ran into a couple of problems using the C-API, so I > suggest looking into Cython instead. It basically translates Python to C > and supports optional static usage of C and C++ data types, so you get all > the native code interaction *and* all the Python interaction and features, > without having to go through the hassle of learning how the Python > internals work (and why they don't work for you). The code it generates is > probably also faster and safer than what you are currently writing (no > offence, just experience from reading and writing a lot of such code). > > http://cython.org/ > > Stefan > > -- > https://mail.python.org/mailman/listinfo/python-list > From jqian at tibco.com Mon Feb 19 16:07:02 2018 From: jqian at tibco.com (Jason Qian) Date: Mon, 19 Feb 2018 16:07:02 -0500 Subject: How to link to python 3.6.4 library on linux ? Message-ID: Hi, I am calling python from a c application. It compiles and works fine on the windows. How do I compile and link it on the linux for Python 3.6.4 ? Under python dir, it only have a static library, /opt/Python-3.6.4*/lib*/*libpython3.6m.a* * If I link to it, I got:* */opt/Python-3.6.4/lib/libpython3.6m.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC* */opt/Python-3.6.4/lib/libpython3.6m.a: could not read symbols: Bad value* *Thanks for the help,* From giuseppecostanzi at gmail.com Mon Feb 19 16:45:23 2018 From: giuseppecostanzi at gmail.com (Beppe) Date: Mon, 19 Feb 2018 13:45:23 -0800 (PST) Subject: [ANN] Biovarase ver 2 Message-ID: <9063a382-69aa-4071-b4e0-a30743dd793c@googlegroups.com> Biovarase has been updated to version 2, The project has been migrated from python 2.7 to python 3.5 Biovarase is an application to manage clinical quality control data. The purpose of Quality Control Assurance in a clinical laboratory is to allow the control of the performances of an analytical procedure showing an alarm as soon as the trial doesn't result in the degree to respect the defined analytical rules. Biovarase furthermore calculates the classical statistical parameters for the quality control assurance ,e.g. sd, cv%, avg, and even the Imp(%), Bias(%) and TEa (total allowable error) using data retrived from: Current databases on biologic variation: pros, cons and progress Scand J Clin Lab Invest 1999;59:491-500. updated with the most recent specifications made available in 2014. It uses even the famous Westgard's rules to monitor results dataset. All the data are managed by SQLite database and matplotlib. To show levey jennings graph, in the main windows select a test and choose the relative batch. To deactivate/activate a result make double click on it. To insert, update or delete a batch or a result open from File/Batchs and results. To export data to a temp excel file click on File/Export. Biovarase requires Python 3 Biovarase use Tkinter and matplotlib All source code on https://github.com/1966bc/Biovarase I'made it for fun :(. I would appreciate any suggestions you might have. regards beppe From rosuav at gmail.com Mon Feb 19 17:02:17 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 09:02:17 +1100 Subject: [ANN] Biovarase ver 2 In-Reply-To: <9063a382-69aa-4071-b4e0-a30743dd793c@googlegroups.com> References: <9063a382-69aa-4071-b4e0-a30743dd793c@googlegroups.com> Message-ID: On Tue, Feb 20, 2018 at 8:45 AM, Beppe wrote: > > Biovarase has been updated to version 2, > > The project has been migrated from python 2.7 to python 3.5 > > Biovarase is an application to manage clinical quality control data. > > The purpose of Quality Control Assurance in a clinical laboratory is to allow the control of the performances of an analytical procedure showing an alarm as soon as the trial doesn't result in the degree to respect the defined analytical rules. Biovarase furthermore calculates the classical statistical parameters for the quality control assurance ,e.g. sd, cv%, avg, and even the Imp(%), Bias(%) and TEa (total allowable error) using data retrived from: Current databases on biologic variation: pros, cons and progress Scand J Clin Lab Invest 1999;59:491-500. updated with the most recent specifications made available in 2014. > It uses even the famous Westgard's rules to monitor results dataset. > All the data are managed by SQLite database and matplotlib. > To show levey jennings graph, in the main windows select a test and choose the relative batch. > To deactivate/activate a result make double click on it. > To insert, update or delete a batch or a result open from File/Batchs and results. > To export data to a temp excel file click on File/Export. > > Biovarase requires Python 3 > Biovarase use Tkinter and matplotlib > > All source code on > > https://github.com/1966bc/Biovarase > > I'made it for fun :(. > I would appreciate any suggestions you might have. Cool! Good to see. A few small suggestions, but nothing major. All your git commits are "file upload" apart from the occasional web edit; I would recommend getting familiar with command-line git and making commits with useful messages. (I was hoping to have a look at the changes that you did for Py2 -> Py3, but couldn't find them in the pile of big changes.) I'd also suggest getting an actual LICENSE or COPYING file, since your README says your code is GPL'd. And you may want to look into the formatting of your README - there are a few things that probably should be turned into bulleted lists rather than flowing as paragraphs. But otherwise, cool! I like to see this sort of thing published and open sourced. ChrisA From rosuav at gmail.com Mon Feb 19 17:04:37 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 09:04:37 +1100 Subject: How to link to python 3.6.4 library on linux ? In-Reply-To: References: Message-ID: On Tue, Feb 20, 2018 at 8:07 AM, Jason Qian via Python-list wrote: > Hi, > > I am calling python from a c application. > It compiles and works fine on the windows. How do I compile and link > it on the linux for Python 3.6.4 ? > > Under python dir, it only have a static library, > > /opt/Python-3.6.4*/lib*/*libpython3.6m.a* > > * If I link to it, I got:* > > */opt/Python-3.6.4/lib/libpython3.6m.a(abstract.o): relocation R_X86_64_32S > against `_Py_NotImplementedStruct' can not be used when making a shared > object; recompile with -fPIC* > */opt/Python-3.6.4/lib/libpython3.6m.a: could not read symbols: Bad value* > By "calling Python from C", do you mean what the docs call "embedding"? https://docs.python.org/3/extending/embedding.html I would recommend following the tutorial there, if you haven't already. If you have, and the tutorial code doesn't work for you, post your code and what you did, and we'll try to help with that. ChrisA From giuseppecostanzi at gmail.com Mon Feb 19 17:21:28 2018 From: giuseppecostanzi at gmail.com (Beppe) Date: Mon, 19 Feb 2018 14:21:28 -0800 (PST) Subject: [ANN] Biovarase ver 2 In-Reply-To: References: <9063a382-69aa-4071-b4e0-a30743dd793c@googlegroups.com> Message-ID: Il giorno luned? 19 febbraio 2018 23:02:43 UTC+1, Chris Angelico ha scritto: > On Tue, Feb 20, 2018 at 8:45 AM, Beppe wrote: > > > > Biovarase has been updated to version 2, > > > > The project has been migrated from python 2.7 to python 3.5 > > > > Biovarase is an application to manage clinical quality control data. > > > > The purpose of Quality Control Assurance in a clinical laboratory is to allow the control of the performances of an analytical procedure showing an alarm as soon as the trial doesn't result in the degree to respect the defined analytical rules. Biovarase furthermore calculates the classical statistical parameters for the quality control assurance ,e.g. sd, cv%, avg, and even the Imp(%), Bias(%) and TEa (total allowable error) using data retrived from: Current databases on biologic variation: pros, cons and progress Scand J Clin Lab Invest 1999;59:491-500. updated with the most recent specifications made available in 2014. > > It uses even the famous Westgard's rules to monitor results dataset. > > All the data are managed by SQLite database and matplotlib. > > To show levey jennings graph, in the main windows select a test and choose the relative batch. > > To deactivate/activate a result make double click on it. > > To insert, update or delete a batch or a result open from File/Batchs and results. > > To export data to a temp excel file click on File/Export. > > > > Biovarase requires Python 3 > > Biovarase use Tkinter and matplotlib > > > > All source code on > > > > https://github.com/1966bc/Biovarase > > > > I'made it for fun :(. > > I would appreciate any suggestions you might have. > > Cool! Good to see. > > A few small suggestions, but nothing major. All your git commits are > "file upload" apart from the occasional web edit; I would recommend > getting familiar with command-line git and making commits with useful > messages. (I was hoping to have a look at the changes that you did for > Py2 -> Py3, but couldn't find them in the pile of big changes.) I'd > also suggest getting an actual LICENSE or COPYING file, since your > README says your code is GPL'd. And you may want to look into the > formatting of your README - there are a few things that probably > should be turned into bulleted lists rather than flowing as > paragraphs. But otherwise, cool! I like to see this sort of thing > published and open sourced. > > ChrisA Well, I don't manage a lot of git hub yet but I will surely keep in mind of your suggestions. Publishing this project and others is the least one that I can do for thanking the python's programmer community of the things that I have learned ...I believe even that I must also work hard on my English.... thank you very much for your attention Chris From best_lay at yahoo.com Mon Feb 19 18:01:33 2018 From: best_lay at yahoo.com (Wildman) Date: Mon, 19 Feb 2018 17:01:33 -0600 Subject: Python 2 to 3 Conversion References: Message-ID: On Tue, 20 Feb 2018 05:39:15 +1100, Chris Angelico wrote: > On Tue, Feb 20, 2018 at 3:53 AM, Wildman via Python-list > wrote: >> On Tue, 20 Feb 2018 02:26:19 +1100, Chris Angelico wrote: >> >>> * Opaque IOCTLs >> >> Would you mind to elaborate a little about your >> concerns? > > Look at your original code: it's impossible to figure out what it's > doing or whether it's doing it correctly. "Opaque" in this context > means that you can't grok the code without external help. This is a > code smell, and a strong indication that you're "fiddling" in stuff > way lower level than you normally want to be working with. It's > usable, but it's dirty and leaves everything up to you (do you know > for sure, for instance, if this is backward compatible? > Cross-platform?), and that makes it comparable to depending on an > external program. > > ChrisA I understand and I agree for the most part. It does concern me that I don't understand the code. Anytime in Linux you start poking around things at kernel level, you need your eyes to be wide open. However, to me it looks like the code is only quarying ioctl and not trying to change any- thing. If this is true, there is no problem. For the issue of backward compatibility, there are good people on a few of the Linux newsgroups that are willing to test code snippets for me. I can get a wide range of distros, desktops and kernel versions. And of course different python3 versions. I know it is impossible to write code that will be guaranteed to run on all conditions. But the test results I get will or should let me know right off if there are glaring problems. As far as cross-platform goes, that is not an issue. Every thing I do is targeted for the Linux platform. I will keep "ip" in mind as a backup plan. As a novice Python programmer, I am looking at the amount of code it will take to parse ip's output compared to the few lines of code it takes to call ioctl. I really appreciate your insight. Thank you. -- GNU/Linux user #557453 "There are only 10 types of people in the world... those who understand Binary... and those who don't." -Spike From best_lay at yahoo.com Mon Feb 19 18:09:58 2018 From: best_lay at yahoo.com (Wildman) Date: Mon, 19 Feb 2018 17:09:58 -0600 Subject: Python 2 to 3 Conversion References: Message-ID: On Tue, 20 Feb 2018 05:31:27 +1100, Chris Angelico wrote: > On Tue, Feb 20, 2018 at 3:49 AM, Wildman via Python-list > wrote: >> On Mon, 19 Feb 2018 12:32:49 +0000, Rhodri James wrote: >> >>> On 18/02/18 16:18, Wildman via Python-list wrote: >>>>> But that's only going to show one (uplink) address. If I needed to get >>>>> ALL addresses for ALL network adapters, I'd either look for a library, >>>>> and if one wasn't easily found, I'd shell out to the "ip" command and >>>>> parse its output.:) >>>>> >>>> I considered using the "ip" command but I prefer not to >>>> depend on external programs if I can get around it. I >>>> know that might be considered silly but that's just me. >>> >>> It's not silly at all, but for Linux networking it might be the best >>> idea. I believe in theory you are supposed to use libnl (in some >>> suitable wrapping), but my experience with that is that it is badly >>> documented and horrendously unreliable. >> >> It looks like libnl would do what I want but there is >> a problem. When finished, my program will be released >> in the form of a Debian (*.deb) package and used by, >> for the most part, 'average' Linux users. These are >> people that know their way around Linux but know >> nothing or very little about Python. Installing a >> package using pip by them is pretty much out of the >> question. The system's package manager must be able >> to handle the program's dependencies so third-party >> packages are out of the question. >> >> But thanks for the suggestion. > > If you're distributing it as a .deb, you don't want to use pip to grab > additional libraries. Ideally, you'd want the library to *also* be > available through the package manager, which would mean you can simply > record it as a dependency. If it's not... it's not gonna be easy. > > ChrisA Yes, you are correct. Third-party pip packages are always a no-no. Speaking of which, there is a library called Netifaces that will easily do exactly what I want with a few lines of code. But, it is not to be found in any Linux distro's repository that I can find. Oh well... -- GNU/Linux user #557453 "There are only 10 types of people in the world... those who understand Binary... and those who don't." -Spike From rosuav at gmail.com Mon Feb 19 18:55:28 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 10:55:28 +1100 Subject: Python 2 to 3 Conversion In-Reply-To: References: Message-ID: On Tue, Feb 20, 2018 at 10:09 AM, Wildman via Python-list wrote: > Yes, you are correct. Third-party pip packages are always > a no-no. > > Speaking of which, there is a library called Netifaces that > will easily do exactly what I want with a few lines of code. > But, it is not to be found in any Linux distro's repository > that I can find. Oh well... > $ apt-cache search netifaces python-netifaces - portable network interface information - Python 2.x python-netifaces-dbg - portable network interface information - Python 2.x debug extension python3-netifaces - portable network interface information - Python 3.x python3-netifaces-dbg - portable network interface information - Python 3.x debug extension $ lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 9.3 (stretch) Release: 9.3 Codename: stretch The given homepage URL is http://alastairs-place.net/projects/netifaces/ - is that the right one? ChrisA From best_lay at yahoo.com Mon Feb 19 20:05:52 2018 From: best_lay at yahoo.com (Wildman) Date: Mon, 19 Feb 2018 19:05:52 -0600 Subject: Python 2 to 3 Conversion References: Message-ID: On Tue, 20 Feb 2018 10:55:28 +1100, Chris Angelico wrote: > The given homepage URL is > http://alastairs-place.net/projects/netifaces/ - is that the right > one? > > ChrisA Yes, that is the right one. Now I'm feeling a little stupid. I should have remembered that many python library package names start with python- or python3-. Problem solved. A thousand thank-you's. -- GNU/Linux user #557453 The cow died so I don't need your bull! From Richard at Damon-Family.org Mon Feb 19 20:26:20 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Mon, 19 Feb 2018 20:26:20 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On 2/19/18 10:35 AM, Chris Angelico wrote: > On Tue, Feb 20, 2018 at 12:34 AM, Steven D'Aprano > wrote: >> On Mon, 19 Feb 2018 20:14:32 +1100, Chris Angelico wrote: >> >>> As an integer, 3.141590 is 1078530000 $ >>> >>> Looks to me like C is perfectly happy to interpret a float as an int. >> Yes, but that's not an *automatic* coercion. To count as weakly typed, >> the compiler has to do it automatically, without an explicit cast or >> conversion. > Fair enough. If you ignore warnings, then C does have that kind of weak typing: > > $ cat demo.c > #include > > int main() { > float f = 3.14159; > int *i = &f; > printf("As an integer, %f is %d\n", f, *i); > return 0; > } > > $ gcc demo.c > demo.c: In function ?main?: > demo.c:5:14: warning: initialization from incompatible pointer type > [-Wincompatible-pointer-types] > int *i = &f; > ^ > $ > > GCC was quite happy to compile that code, even though the type of "&f" > is "pointer to float", and it's being assigned to a variable of type > "pointer to int". But C is a language saddled with so much history and > backward compatibility constraints that there are some things you just > CAN'T make into errors; so in terms of "how safe is C?", I'd have to > say that warnings (especially those that are enabled by default - I > didn't need to say "-Wall" for this test), count as "disallowing", > especially if all the major compilers emit warnings on the same code. > But I do see the argument that "it compiles, so the language clearly > permits it". > > ChrisA One thing to note, that is more an issue with GCC than with C. By the standard, that is a constraint violation, that requires a diagnostic, and the C standard says it provides no definition of what should happen here, which is about as strong as it gets to defining something as an 'Error', the only thing with a stronger requirement is the #error statement which must not running the program. GCC has decided that this diagnostic will be considered just a 'Warning' and provides its own meaning to the statement. You really want to run with pedantic-errors enabled to get GCC to reject code with constraint violations. -- Richard Damon From rosuav at gmail.com Mon Feb 19 20:29:30 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 12:29:30 +1100 Subject: Python 2 to 3 Conversion In-Reply-To: References: Message-ID: On Tue, Feb 20, 2018 at 12:05 PM, Wildman via Python-list wrote: > On Tue, 20 Feb 2018 10:55:28 +1100, Chris Angelico wrote: > >> The given homepage URL is >> http://alastairs-place.net/projects/netifaces/ - is that the right >> one? >> >> ChrisA > > Yes, that is the right one. Now I'm feeling a little stupid. > I should have remembered that many python library package > names start with python- or python3-. > > Problem solved. A thousand thank-you's. > No problem, this is why we don't work alone :) If you can work with that on all the platforms you need, awesome! The dirtiness of calling a subprocess or an opaque IOCTL can be wrapped up nicely inside a library. Always an improvement IMO. ChrisA From jqian at tibco.com Mon Feb 19 22:04:22 2018 From: jqian at tibco.com (Jason Qian) Date: Mon, 19 Feb 2018 22:04:22 -0500 Subject: How to link to python 3.6.4 library on linux ? In-Reply-To: References: Message-ID: Thanks Chris, I think I figured it out that when build python on Linux, we need to enable-shared. Thanks again, On Mon, Feb 19, 2018 at 5:04 PM, Chris Angelico wrote: > On Tue, Feb 20, 2018 at 8:07 AM, Jason Qian via Python-list > wrote: > > Hi, > > > > I am calling python from a c application. > > It compiles and works fine on the windows. How do I compile and > link > > it on the linux for Python 3.6.4 ? > > > > Under python dir, it only have a static library, > > > > /opt/Python-3.6.4*/lib*/*libpython3.6m.a* > > > > * If I link to it, I got:* > > > > */opt/Python-3.6.4/lib/libpython3.6m.a(abstract.o): relocation > R_X86_64_32S > > against `_Py_NotImplementedStruct' can not be used when making a shared > > object; recompile with -fPIC* > > */opt/Python-3.6.4/lib/libpython3.6m.a: could not read symbols: Bad > value* > > > > By "calling Python from C", do you mean what the docs call "embedding"? > > https://docs.python.org/3/extending/embedding.html > > I would recommend following the tutorial there, if you haven't > already. If you have, and the tutorial code doesn't work for you, post > your code and what you did, and we'll try to help with that. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From rantingrickjohnson at gmail.com Mon Feb 19 22:09:35 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 19 Feb 2018 19:09:35 -0800 (PST) Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: Message-ID: <7108621d-3377-478e-aaf2-bc95805816ea@googlegroups.com> On Friday, February 16, 2018 at 10:25:32 PM UTC-6, Chris Angelico wrote: [...] > This is often touted as a necessity for industrial-grade > software. It isn't. There are many things that a type > system, no matter how sophisticated, cannot catch; for some > reason, though, we don't hear people saying "C is useless > for industrial-grade software because it doesn't have > function contracts". And likewise, there are many problems that a seatbelt cannot protect you against -- like for instance: a car-jacker shooting you in the head at point-blank range. But we don't throw the baby out with the bath water, do we? > Anyway, if you want some system of type checking, you can > use static analysis (eg tools like MyPy) to go over your > code the same way a compiler might. Are you suggesting this project for std-lib inclusion? > "The first glaring issue is that I have no guarantee that > is_valid() returns a bool type." -- huh? It's being used in > a boolean context, and it has a name that starts "is_". How > much guarantee are you looking for? *ANY* object can be > used in an 'if', so it doesn't even matter. This is a > stupidly contrived criticism. > > > Python markets itself as a dynamically-typed programming > > language, similar to Perl, Ruby, or JavaScript. The word > > ?dynamically typed? has a positive connotation, more so > > than ?weakly typed.? Conversely, the word ?strongly typed? > > sounds more positive than saying ?statically typed.? > > Those are all different terms, and all of them are > misunderstood. They're not synonyms. The author was underscoring the unfortunate psychological semantics at play when words like "strong" are juxtaposed with words like "weak". Sadly, old boy, i believe you've missed the point! > > Python ships with a threading module; but due to how the > > Python interpreter is implemented, it can only ever run > > one thread at a time. This is due to the infamous Global > > Interpreter Lock (GIL). In an age when the only computers > > around had a single core, this was nothing to fuss over. > > Totally not true. The GIL does not stop other threads from > running. Also, Python has existed for multiple CPU systems > pretty much since its inception, I believe. (Summoning the > D'Aprano for history lesson?) Unfortunately "D'App-ran?...Oh!" is still debugging his hello world program. He'll be along later. Or never... Whichever comes /last/. [...] From breamoreboy at gmail.com Mon Feb 19 22:16:49 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 19 Feb 2018 19:16:49 -0800 (PST) Subject: Gmane seems to be gone In-Reply-To: References: Message-ID: <4c676151-811f-4241-a171-6776634c5222@googlegroups.com> On Sunday, February 18, 2018 at 8:23:03 PM UTC, Mark Lawrence wrote: > On 18/02/18 18:03, Grant Edwards wrote: > > On 2018-02-18, Dennis Lee Bieber wrote: > >> On Sun, 18 Feb 2018 17:26:54 +0000 (UTC), Grant Edwards > >> declaimed the following: > >> > >>> > >>> It was Yomura who picked up the archive and continued the gateway > >>> service a couple years ago, and they just renewed the domain for > >>> another year, so perhaps it's just a temporary DNS problem? > >> > >> Looks like the DNS update is propagating -- I can now reach gmane; but > >> it hasn't shown any new traffic on any of the groups/lists I follow. > > > > Yay! > > > > Gmane.org just started resolving again, and I'm posting this followup > > from slrn via news.gmane.org. I really should donate some money to > > gmane's operators... > > > > Great stuff :) Of course as gmane wasn't working I posted earlier today > on gg but I believe you've opted out of that so wouldn't have seen it :( > What the hell, normal service is resumed :) > > -- > My fellow Pythonistas, ask not what our language can do for you, ask > what you can do for our language. > > Mark Lawrence Blast, seems like it's down again. Can somebody confirm that I'm correct please, cheers :( -- Kindest regards. Mark Lawrence. From rantingrickjohnson at gmail.com Mon Feb 19 22:28:20 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 19 Feb 2018 19:28:20 -0800 (PST) Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <87zi48yuec.fsf@nightsong.com> References: <85zi48b6r5.fsf@benfinney.id.au> <87zi48yuec.fsf@nightsong.com> Message-ID: On Saturday, February 17, 2018 at 12:58:49 AM UTC-6, Paul Rubin wrote: [...] > Beyond that, the Python community (with some exceptions) seems to have a > widespread hatred of threads. It instead prefers to handle concurrent > i/o with in-thread async schemes that the rest of the world left behind > in the 1960s. Hmm... and many wonder if the source of that hatred is a direct result of CPython's unholy alliance with the GIL? From mcohenact at gmail.com Tue Feb 20 00:08:17 2018 From: mcohenact at gmail.com (Marc Cohen) Date: Mon, 19 Feb 2018 21:08:17 -0800 (PST) Subject: could use some help with this problem! I've been working on it for days but cant seem to get it right ! Message-ID: <8a1de4c4-915a-4147-89f1-faca7ef62b0c@googlegroups.com> USING PYTHON 2: Write a program to play this game. This may seem tricky, so break it down into parts. Like many programs, we have to use nested loops (one loop inside another). In the outermost loop, we want to keep playing until we are out of stones. Inside that, we want to keep alternating players. You have the option of either writing two blocks of code, or keeping a variable that tracks the current player. The second way is slightly trickier since we haven't learned lists yet, but it's definitely do-able! Finally, we might want to have an innermost loop that checks if the user's input is valid. Is it a number? Is it a valid number (e.g. between 1 and 5)? Are there enough stones in the pile to take off this many? If any of these answers are no, we should tell the user and re-ask them the question So, the basic outline of the program should be something like this: totalStones = 100 maxStones = 5 pile = TOTAL # all stones are in the pile to start while [pile is not empty]: while [player 1's answer is not valid]: [ask player 1] [execute player1?s move] Do the same for player 2?. (this can be achieved by a for loop) Note how the important numbers 100 and 5 are stored in a single variable at the top. This is good practice -- it allows you to easily change the constants of a program. For example, for testing, you may want to start with only 15 or 20 stones. Be careful with the validity checks. Specifically, we want to keep asking player 1 for their choice as long as their answer is not valid, BUT we want to make sure we ask them at least ONCE. So, for example, we will want to keep a variable which tracks whether their answer is valid, and set it to False initially. Hint: the final number of stones can?t be negative. The game automatically stops when the number of stones reach zero. From breamoreboy at gmail.com Tue Feb 20 00:13:21 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 19 Feb 2018 21:13:21 -0800 (PST) Subject: could use some help with this problem! I've been working on it for days but cant seem to get it right ! In-Reply-To: <8a1de4c4-915a-4147-89f1-faca7ef62b0c@googlegroups.com> References: <8a1de4c4-915a-4147-89f1-faca7ef62b0c@googlegroups.com> Message-ID: <27b0a278-2f6d-45e1-a2c0-2ea892141b82@googlegroups.com> On Tuesday, February 20, 2018 at 5:08:49 AM UTC, Marc Cohen wrote: > USING PYTHON 2: > > Write a program to play this game. This may seem tricky, so break it down into parts. Like many programs, we have to use nested loops (one loop inside another). In the outermost loop, we want to keep playing until we are out of stones. > > Inside that, we want to keep alternating players. You have the option of either writing two blocks of code, or keeping a variable that tracks the current player. The second way is slightly trickier since we haven't learned lists yet, but it's definitely do-able! > > Finally, we might want to have an innermost loop that checks if the user's input is valid. Is it a number? Is it a valid number (e.g. between 1 and 5)? Are there enough stones in the pile to take off this many? If any of these answers are no, we should tell the user and re-ask them the question > > So, the basic outline of the program should be something like this: > > totalStones = 100 > > maxStones = 5 > > pile = TOTAL # all stones are in the pile to start > > while [pile is not empty]: > > while [player 1's answer is not valid]: > > [ask player 1] > > [execute player1?s move] > > Do the same for player 2?. (this can be achieved by a for loop) > > Note how the important numbers 100 and 5 are stored in a single variable at the top. This is good practice -- it allows you to easily change the constants of a program. For example, for testing, you may want to start with only 15 or 20 stones. > > Be careful with the validity checks. Specifically, we want to keep asking player 1 for their choice as long as their answer is not valid, BUT we want to make sure we ask them at least ONCE. So, for example, we will want to keep a variable which tracks whether their answer is valid, and set it to False initially. > > Hint: the final number of stones can?t be negative. The game automatically stops when the number of stones reach zero. Sorry old bean but we don't write code for you. So you show us the code that you've been working on for days and tell us what's wrong with it, and we'll point you in the right direction. -- Kindest regards. Mark Lawrence. From breamoreboy at gmail.com Tue Feb 20 00:25:01 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 19 Feb 2018 21:25:01 -0800 (PST) Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: <2fdec7f8-4ae5-4b19-b9af-cdd31e7e5889@googlegroups.com> On Monday, February 19, 2018 at 1:07:02 PM UTC, Anders Wegge Keller wrote: > P? Mon, 19 Feb 2018 04:39:31 +0000 (UTC) > Steven D'Aprano skrev: > > On Mon, 19 Feb 2018 04:26:32 +0100, Anders Wegge Keller wrote: > > > > > P? Mon, 19 Feb 2018 08:47:14 +1100 > > > Tim Delaney skrev: > > >> On 18 February 2018 at 22:55, Anders Wegge Keller > > >> wrote: > > > > > > > > [...] > > > > > >> You couldn't have got the above much more wrong. > > > > > >> As others have said, typing is about how the underlying memory is > > >> treated. > > > > > > And that is exactly my point. Python does not have a typed list. It > > > have a list that takes whatever is thrown into it. > > > > > > I'll skip the rest, as you totally missed the point. > > > > I think its actually you have totally missed the point. What you want is > > a homogeneous list, a list which only accepts items of the same type. The > > built-in list is not that data structure, but Python already has > > something similar: see the array module. > > Given that I'm the one making a point about the unwarranted smugness, I > know that I'm not missing anything. Array is not even close to providing a > strongly typed container. For all of the yakking about other languages > weak, blah, BCPL, blah, I still wonder where that need to feel superior to > anything else comes from. > > Python isn't particular strong typed. In fact, apart from asking an object > what type it is, types are not that important. It's the interface that > matters. I wonder why this is a sore point for Python developers? > > > -- > //Wegge Congratulations, you're the first new person I've had on my Dream Team for some months, fresh blood is always so welcome. -- Kindest regards. Mark Lawrence. From steve+comp.lang.python at pearwood.info Tue Feb 20 02:54:04 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 07:54:04 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <87eflhv0d2.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On Tue, 20 Feb 2018 02:35:37 +1100, Chris Angelico wrote: > But C is a language saddled with so much history and backward > compatibility constraints that there are some things you just CAN'T make > into errors Indeed. C is not even close to a safe language, hence the move to create (and more importantly, convince people to use!) systems languages like Rust and D which combine C's ability to generate fast code without (unnecessarily) compromising type safety. -- Steve From antoon.pardon at vub.be Tue Feb 20 05:47:15 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 20 Feb 2018 11:47:15 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> Message-ID: <2d78469f-40e6-5068-41f3-5eeaa2e7d2dd@vub.be> On 19-02-18 16:18, Ned Batchelder wrote: > On 2/19/18 9:54 AM, Steven D'Aprano wrote: >> On Mon, 19 Feb 2018 13:28:26 +0000, Paul Moore wrote: >> >>> [1] The most basic question, which people making such claims often >>> can't >>> answer, is "Do you mean that values are strongly typed, or that names >>> are? Or did you mean that variables are, because if so Python doesn't >>> even have variables in the sense that you mean" Programming language >>> semantics are complex. >> An excellent point. >> >> The distinction between typing *values* and *names* is a good one. >> >> >> > > I guess I'll have to continue to grit my teeth as people say, "Python > doesn't have variables."?? Why can't we say, "Python's variables work > differently than other languages"? > > --Ned. Which other languages. It seems python variables seem to work just like lisp, scheme and smalltalk. -- Antoon From ned at nedbatchelder.com Tue Feb 20 06:04:58 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 20 Feb 2018 06:04:58 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <2d78469f-40e6-5068-41f3-5eeaa2e7d2dd@vub.be> References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> <2d78469f-40e6-5068-41f3-5eeaa2e7d2dd@vub.be> Message-ID: <3817b084-0be8-29f0-2cf5-f82f31c98f1f@nedbatchelder.com> On 2/20/18 5:47 AM, Antoon Pardon wrote: > On 19-02-18 16:18, Ned Batchelder wrote: >> On 2/19/18 9:54 AM, Steven D'Aprano wrote: >>> On Mon, 19 Feb 2018 13:28:26 +0000, Paul Moore wrote: >>> >>>> [1] The most basic question, which people making such claims often >>>> can't >>>> answer, is "Do you mean that values are strongly typed, or that names >>>> are? Or did you mean that variables are, because if so Python doesn't >>>> even have variables in the sense that you mean" Programming language >>>> semantics are complex. >>> An excellent point. >>> >>> The distinction between typing *values* and *names* is a good one. >>> >>> >>> >> I guess I'll have to continue to grit my teeth as people say, "Python >> doesn't have variables."?? Why can't we say, "Python's variables work >> differently than other languages"? >> >> --Ned. > Which other languages. It seems python variables seem to work just like > lisp, scheme and smalltalk. > C. That is one of the reasons for my frustration: the "Python has no variables" slogan is incredibly C-centric.? I understand the importance of C, and in the early days of Python, there was good reason to assume that people were familiar with C.? These days, there are far more mainstream languages that work just as Python does. To your "work just like" list we can add Javascript, Ruby, PHP, and Java (for objects, not primitives). --Ned. From antoon.pardon at vub.be Tue Feb 20 06:18:47 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 20 Feb 2018 12:18:47 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> On 19-02-18 15:25, Steven D'Aprano wrote: > >> Ones like C++ has already tied itself itself up in knots just doing the >> basics; I'm not sure how it would handle even my 1,3,5,7,9 type. >> >> But Python has classes and can do some of this stuff; how would it >> handle a numeric type that is constrained to be whole numbers within >> 0..9 inclusive? > This becomes easy at run-time: > > class Digit(int): > def __new__(cls, arg): > instance = super().__new__(cls, arg) > if not 0 <= instance <= 9: > raise ValueError('argument is out of range') > return instance > > The above is obviously not a full-blown production-ready class. But it > illustrates the basic concept. This is the sort of thing that dynamic > languages excel at: enforcing constraints at run-time which are hard to > enforce at compile-time. I don't see how dynamic languages are excelling here. Writing code that ensures a number of constraints can be done just as easily in a static language. Personnally I would prefer the type system of Pascal and Modula2 with their interval type above a Digit class in python. For the simple reason that once you had declared a variable like this: ??? x: 1 .. 10; Each assignment to x would then implicitly do something like an assert to checks the constraint, so it would be impossible to ever assign 11 to x, without an error being thrown. There is no such possibility in Python. You can off course start with x = Digit(5), but the language won't stop you from doing x = 11 later. I'm not proficient with C++, but IIUC, you could make a class in C++ and have the constructor and copy operator check for these kind of things. True it would be run-time checks but that would already be more than Python can give. -- Antoon. From p.f.moore at gmail.com Tue Feb 20 07:11:57 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 20 Feb 2018 12:11:57 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 20 February 2018 at 11:18, Antoon Pardon wrote: > Personnally I would prefer the type system of Pascal and Modula2 with > their interval type > above a Digit class in python. For the simple reason that once you had > declared a variable > like this: > x: 1 .. 10; > > Each assignment to x would then implicitly do something like an assert > to checks the constraint, > so it would be impossible to ever assign 11 to x, without an error being > thrown. > > There is no such possibility in Python. You can off course start with x > = Digit(5), but the language > won't stop you from doing x = 11 later. All that is saying is that in Pascal, variables have types, whereas in Python values have types but variables (names ;-)) don't. It's true, but not particularly important in toy examples like this. In larger scale programs, tracking the "type" of what gets assigned to a variable can be really useful, and carefully managed types can help with this. Pascal/Modula2 (and C/C++) have the compiler do this, Python has a separate tool (MyPy). Early C compilers used to have linters that did some checks outside of the compiler - maybe someday Python will build MyPy into the compiler (although I suspect not). Also, tracking "types" is only half the battle. There are probably very few people who have never mistakenly assigned NULL to a pointer in C that shouldn't be null. Or even in Java, which is supposed to not allow things like that. And tracking const types in C is widely acknowledged to be a mixed blessing at best. Where does "type" end and "documented constraint or programming contract" start? > I'm not proficient with C++, but IIUC, you could make a class in C++ and > have the constructor and > copy operator check for these kind of things. True it would be run-time > checks but that would > already be more than Python can give. That (run-time checks) is exactly the same as Python gives. In C++, variables have types so you could ensure that you could only assign a value of your digit type to a given variable, but as I said above that's a separate point. It's somewhat unrelated (scoping is a different topic than assignment) but you can do { int x = 2; { char *x = "hello"; } } in C, so names can have different types even in C (it's just variables, or names within a specific scope, that have types associated with them). Summary: Different programming languages have different semantics. Which really isn't that surprising... Paul From rosuav at gmail.com Tue Feb 20 07:12:23 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 20 Feb 2018 23:12:23 +1100 Subject: Python on Android? In-Reply-To: <20180218175734.6e8908d9@orbit.fritz.box> References: <20180218175734.6e8908d9@orbit.fritz.box> Message-ID: On Mon, Feb 19, 2018 at 3:57 AM, Johannes Findeisen wrote: > On Sun, 18 Feb 2018 20:57:02 +1100 > Chris Angelico wrote: > >> Does anyone have experience with running Python scripts on Android >> phones? I have a brother (honestly! I'm not actually using a phone >> myself!) who's trying to run one of my scripts in QPython, which >> claims to be version 3.2.2. I think that really truly is a Python 3.2 >> implementation - probing for newer features suggests that it actually >> doesn't even support the u"..." syntax that came (back) in with Python >> 3.3. So... does anyone know of a Python interpreter that's compatible >> with 3.4 or better and runs on Android? >> > > There is an App for Android called "Pydroid 3". You can find it in > the Google Play Store [0]. It provides a Python interpreter in version > 3.6.2 in its current release. > > The Python binary is installed under > > /data/user/0/ru.iiec.pydroid3/files/arm-linux-androideabi/bin/python > > but I can not access it in a normal terminal without being user "root" > but it is usable from within the terminal included in the Pydroid App. Many thanks for all the suggestions. Pydroid is the one that has worked; he's now able to run stuff happily from the terminal. Next step is to create an icon that invokes it, but that's looking rather harder (probably means I'm going to have to learn Kivy, and I already have no shortage of projects...). Everything's working in the terminal (even a read-write $HOME, which we didn't have previously), and I'm pretty happy with that! All suggestions were read and are appreciated, even though I'm only quoting the one we ended up going with. Thank you Kirill, MRAB, Abdur, and Johannes! ChrisA From steve+comp.lang.python at pearwood.info Tue Feb 20 07:24:39 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 12:24:39 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On Tue, 20 Feb 2018 12:18:47 +0100, Antoon Pardon wrote: > On 19-02-18 15:25, Steven D'Aprano wrote: >> >>> Ones like C++ has already tied itself itself up in knots just doing >>> the basics; I'm not sure how it would handle even my 1,3,5,7,9 type. >>> >>> But Python has classes and can do some of this stuff; how would it >>> handle a numeric type that is constrained to be whole numbers within >>> 0..9 inclusive? >> This becomes easy at run-time: >> >> class Digit(int): >> def __new__(cls, arg): >> instance = super().__new__(cls, arg) >> if not 0 <= instance <= 9: >> raise ValueError('argument is out of range') >> return instance >> >> The above is obviously not a full-blown production-ready class. But it >> illustrates the basic concept. This is the sort of thing that dynamic >> languages excel at: enforcing constraints at run-time which are hard to >> enforce at compile-time. > > I don't see how dynamic languages are excelling here. Writing code that > ensures a number of constraints can be done just as easily in a static > language. *Just* as easily? Really? Can you tell us which languages can enforce at compile-time that integer N is a multiple of four but not a multiple of 100 unless it is also a multiple of 400? (The same constraint as leap years.) What type declaration would you write for that? Its easy to make overblown claims about how great static typing is, but people didn't invent dynamically typed languages because they wanted to program in a worse language. They did it because dynamic typing allows us to do things which are hard, or impossible, in statically typed languages. > Personnally I would prefer the type system of Pascal and Modula2 with > their interval type > above a Digit class in python. For the simple reason that once you had > declared a variable > like this: > ??? x: 1 .. 10; I agree that Pascal-style interval types are nice, but they're also limited to intervals, and *integer* intervals at that. Pascal doesn't support floating point ranges, such as "x is a Real between 0 and 1". How would you write a type declaration for numbers like these: 4, 6, 8, 24, 26, 48, 50, 124, 126, 342, 344, ... 3909821048582988050 in the language of your choice? Yes, there is a pattern: one less than, and one more than, powers of 5 and 7, up to a maximum of 2**64. That is, 5?1, 25?1, 125?1, ... 7?1, 49?1, 343?1, ... Its one thing to say that a type system could enforce this at compile- time. Its another to demonstrate an existing type system which actually does. > Each assignment to x would then implicitly do something like an assert > to checks the constraint, > so it would be impossible to ever assign 11 to x, without an error being > thrown. As I remember it, Pascal range checking is just dynamic typing in disguise. Statically, x is just an integer, with no interval checking performed at compile-time except for assignment by literals. Everything else is run-time range checking. By which I mean, the compiler would complain if you tried: var x: 1 .. 10; begin x := 11; end; but it would allow this: var a, b: integer; x: 1 .. 10; begin a := 10; b := 1; x := a + b; end; However, the second would generate a runtime range check error and halt the program. That at least is how the Lightspeed Pascal (later Think Pascal) compiler on the Apple Mac operated, and as far as I know all other Pascal compilers worked the same. Now that was over 30 years ago, and I daresay that type checkers these days are more sophisticated and powerful than Pascal's pretty simple checker. But still, there's only so much you can do at compile-time. > There is no such possibility in Python. You can off course start with x > = Digit(5), but the language > won't stop you from doing x = 11 later. Of course you're right that Python won't stop you assigning (say) a float to x. On the other hand, Pascal and C won't allow you to assign a string to something which previously was an integer. The need to create new labels for things just to placate the compiler is one of the more annoying things about static typing. Yes, x was a number, now its a string. If *I* can keep track of that, why can't the compiler? > I'm not proficient with C++, but IIUC, you could make a class in C++ and > have the constructor and > copy operator check for these kind of things. True it would be run-time > checks but that would > already be more than Python can give. Sure. Its a different approach, and not without its merits. -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 20 07:28:25 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 12:28:25 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> <20180219163429.048ba054@wegge.dk> Message-ID: On Mon, 19 Feb 2018 16:34:29 +0100, Anders Wegge Keller wrote: > P? Mon, 19 Feb 2018 15:15:19 +0000 (UTC) Steven D'Aprano > skrev: >> On Mon, 19 Feb 2018 14:06:36 +0100, Anders Wegge Keller wrote: >> >> > Array is not even close to providing a strongly typed container. >> >> That's a mighty powerful claim that goes against the documentation for >> the array module. Can you back your claims up? >> >> Here's an array and a list: > > Make me an array of tuples with two integers and a string, and we can > talk. The array module's failure to support the specific type you want has no bearing on whether or not Python is statically typed. The question isn't "does Python provide you with every imaginable homogeneous array type you might ever want?", we already know the answer to that is no. The question is your claim that Python objects aren't strongly typed, and your obvious attempt to avoid answering my challenge is answer enough. If you could meet the challenge, you would have done it. -- Steve From bc at freeuk.com Tue Feb 20 07:43:22 2018 From: bc at freeuk.com (bartc) Date: Tue, 20 Feb 2018 12:43:22 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 20/02/2018 12:11, Paul Moore wrote: > On 20 February 2018 at 11:18, Antoon Pardon wrote: >> There is no such possibility in Python. You can off course start with x >> = Digit(5), but the language >> won't stop you from doing x = 11 later. >> I'm not proficient with C++, but IIUC, you could make a class in C++ and >> have the constructor and >> copy operator check for these kind of things. True it would be run-time >> checks but that would >> already be more than Python can give. I don't know if that counts. In Pascal (and presumably Ada) then all the gubbins need to make this work properly: var x: 1..10; x = 10; x = x + 1; { error? } would already be in place. Trying to emulate that within a language is complex and unwieldy, and someone has to know how to do it, and then do it. If someone else creates an add-on for it, then those tend to be large and cumbersome (because they have to have every conceivable bell and whistle). And it's an extra dependency. The end result: if I wanted to sit down right now and have a variable and/or type in Python or C++ that is constrained to be within 1..10, then I couldn't. I'd have to use a generic integer type. And in fact, in Python, I couldn't even do that, as I can assign anything at all to x. This is not necessarily undesirable: part of the point of dynamic languages is being informal and having less discipline imposed so that things can get done more rapidly. But enforced discipline can also be useful. > It's somewhat unrelated (scoping is a different topic than assignment) > but you can do > > { > int x = 2; > { > char *x = "hello"; > } > } > > in C, so names can have different types even in C (it's just > variables, or names within a specific scope, that have types > associated with them). That's scope; Python has that too. (Although not C's block scopes and its bizarre rules which mean that you can have up 5 different 'x' meanings even within the same block. And an unlimited number of 'x' scopes within the same function. (This is allowed in C: int L; L: L=10; goto L; ) -- bartc From antoon.pardon at vub.be Tue Feb 20 08:04:29 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 20 Feb 2018 14:04:29 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 20-02-18 13:11, Paul Moore wrote: > On 20 February 2018 at 11:18, Antoon Pardon wrote: >> Personnally I would prefer the type system of Pascal and Modula2 with >> their interval type >> above a Digit class in python. For the simple reason that once you had >> declared a variable >> like this: >> x: 1 .. 10; >> >> Each assignment to x would then implicitly do something like an assert >> to checks the constraint, >> so it would be impossible to ever assign 11 to x, without an error being >> thrown. >> >> There is no such possibility in Python. You can off course start with x >> = Digit(5), but the language >> won't stop you from doing x = 11 later. > All that is saying is that in Pascal, variables have types, whereas in > Python values have types but variables (names ;-)) don't. It's true, > but not particularly important in toy examples like this. In larger > scale programs, tracking the "type" of what gets assigned to a > variable can be really useful, and carefully managed types can help > with this. Pascal/Modula2 (and C/C++) have the compiler do this, > Python has a separate tool (MyPy). In Pascal and Modula2 type checking was not limited to compile time. Part of that type checking was done at by the runtime enviroment. >> I'm not proficient with C++, but IIUC, you could make a class in C++ and >> have the constructor and >> copy operator check for these kind of things. True it would be run-time >> checks but that would >> already be more than Python can give. > That (run-time checks) is exactly the same as Python gives. No it isn't exactly the same. Those runtime checks are implicit. They are like a contract made at declaration time, which make it impossible to assign something to the variable that doesn't meet the constraint without the program throwing an error. You can't do something like that in Python. From wegge at wegge.dk Tue Feb 20 08:12:08 2018 From: wegge at wegge.dk (Anders Wegge Keller) Date: Tue, 20 Feb 2018 14:12:08 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> <20180219163429.048ba054@wegge.dk> Message-ID: <20180220141208.6d81b8f0@wegge.dk> P? Tue, 20 Feb 2018 12:28:25 +0000 (UTC) Steven D'Aprano skrev: > On Mon, 19 Feb 2018 16:34:29 +0100, Anders Wegge Keller wrote: > > > P? Mon, 19 Feb 2018 15:15:19 +0000 (UTC) Steven D'Aprano > > skrev: > >> On Mon, 19 Feb 2018 14:06:36 +0100, Anders Wegge Keller wrote: > >> > [...] > >> > >> That's a mighty powerful claim that goes against the documentation for > >> the array module. Can you back your claims up? > >> > >> Here's an array and a list: > > > > Make me an array of tuples with two integers and a string, and we can > > talk. > > The array module's failure to support the specific type you want has no > bearing on whether or not Python is statically typed. You claimed Array could do what I requested, i.e. container type that guaranteed only one type inside. Furthermore, you are misrepresenting C with malice. I don't care why you feel that a simple observation is a personal attack, but I see you behave as a spoiled kid. I haven\t got the time for it, sop go and pout in your corner. -- //Wegge From p.f.moore at gmail.com Tue Feb 20 08:12:31 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 20 Feb 2018 13:12:31 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 20 February 2018 at 13:04, Antoon Pardon wrote: > On 20-02-18 13:11, Paul Moore wrote: >> On 20 February 2018 at 11:18, Antoon Pardon wrote: >>> Personnally I would prefer the type system of Pascal and Modula2 with >>> their interval type >>> above a Digit class in python. For the simple reason that once you had >>> declared a variable >>> like this: >>> x: 1 .. 10; >>> >>> Each assignment to x would then implicitly do something like an assert >>> to checks the constraint, >>> so it would be impossible to ever assign 11 to x, without an error being >>> thrown. >>> >>> There is no such possibility in Python. You can off course start with x >>> = Digit(5), but the language >>> won't stop you from doing x = 11 later. >> All that is saying is that in Pascal, variables have types, whereas in >> Python values have types but variables (names ;-)) don't. It's true, >> but not particularly important in toy examples like this. In larger >> scale programs, tracking the "type" of what gets assigned to a >> variable can be really useful, and carefully managed types can help >> with this. Pascal/Modula2 (and C/C++) have the compiler do this, >> Python has a separate tool (MyPy). > > In Pascal and Modula2 type checking was not limited to compile time. Part > of that type checking was done at by the runtime enviroment. So? What point are you making? (I genuinely don't see - you surely aren't saying that Python can't implement runtime checks, and this is all related to your original comment that "There is no such possibility in Python"). >>> I'm not proficient with C++, but IIUC, you could make a class in C++ and >>> have the constructor and >>> copy operator check for these kind of things. True it would be run-time >>> checks but that would >>> already be more than Python can give. >> That (run-time checks) is exactly the same as Python gives. > > No it isn't exactly the same. Those runtime checks are implicit. They are like > a contract made at declaration time, which make it impossible to assign something > to the variable that doesn't meet the constraint without the program throwing an > error. > > You can't do something like that in Python. I *am* (reasonably) proficient in C++, and (other than the "values have types rather than variables/names" point I already stated was a difference in how the languages work) I don't see anything in what you're describing as "things C++ can do" that Python can't. Sigh. Languages are different. That's my point. What's yours? If it's that Python is worse than (some other language) then so what? Probably true in some cases, but what makes you think you'll get enthusiastic approval for such a statement in a Python group? Bored now. Paul From antoon.pardon at vub.be Tue Feb 20 08:38:41 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 20 Feb 2018 14:38:41 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 20-02-18 13:24, Steven D'Aprano wrote: > On Tue, 20 Feb 2018 12:18:47 +0100, Antoon Pardon wrote: > >> On 19-02-18 15:25, Steven D'Aprano wrote: >>>> Ones like C++ has already tied itself itself up in knots just doing >>>> the basics; I'm not sure how it would handle even my 1,3,5,7,9 type. >>>> >>>> But Python has classes and can do some of this stuff; how would it >>>> handle a numeric type that is constrained to be whole numbers within >>>> 0..9 inclusive? >>> This becomes easy at run-time: >>> >>> class Digit(int): >>> def __new__(cls, arg): >>> instance = super().__new__(cls, arg) >>> if not 0 <= instance <= 9: >>> raise ValueError('argument is out of range') >>> return instance >>> >>> The above is obviously not a full-blown production-ready class. But it >>> illustrates the basic concept. This is the sort of thing that dynamic >>> languages excel at: enforcing constraints at run-time which are hard to >>> enforce at compile-time. >> I don't see how dynamic languages are excelling here. Writing code that >> ensures a number of constraints can be done just as easily in a static >> language. > *Just* as easily? Really? > > Can you tell us which languages can enforce at compile-time that integer > N is a multiple of four but not a multiple of 100 unless it is also a > multiple of 400? (The same constraint as leap years.) Why should this be done at compile time? I say a static language can do the same as a dynamic language and your counter point is to ask for how that static language can do something extra. The point I am making is that you claim dynamic languages are excelling by being less demaning of the dynamic language. So yes the static language can check those kind of restraints at runtime just as easily as a dynamic language. > What type declaration would you write for that? > > Its easy to make overblown claims about how great static typing is, but > people didn't invent dynamically typed languages because they wanted to > program in a worse language. They did it because dynamic typing allows us > to do things which are hard, or impossible, in statically typed languages. Maybe but you didn't give an example of something that is hard of impossible in a statically typed language. There is nothing impossible or hard in checking a number of constraints during runtime in a statically typed language. > I agree that Pascal-style interval types are nice, but they're also > limited to intervals, and *integer* intervals at that. Pascal doesn't > support floating point ranges, such as "x is a Real between 0 and 1". > > How would you write a type declaration for numbers like these: > > 4, 6, 8, 24, 26, 48, 50, 124, 126, 342, 344, ... 3909821048582988050 > > in the language of your choice? > > Yes, there is a pattern: one less than, and one more than, powers of 5 > and 7, up to a maximum of 2**64. > > That is, 5?1, 25?1, 125?1, ... 7?1, 49?1, 343?1, ... Well as far as I know you could do this in C++ by having the constructor and copy-operator make the needed checks. > Its one thing to say that a type system could enforce this at compile- > time. Its another to demonstrate an existing type system which actually > does. I am not talking about compile time. There is nothing that prevents a statically typed language to do some of the checks at runtime. >> Each assignment to x would then implicitly do something like an assert >> to checks the constraint, >> so it would be impossible to ever assign 11 to x, without an error being >> thrown. > As I remember it, Pascal range checking is just dynamic typing in > disguise. Statically, x is just an integer, with no interval checking > performed at compile-time except for assignment by literals. Everything > else is run-time range checking. So? As far as I know Pascal is generally considered a statically typed language. That some of the constraints imposed by the type can only be checked at runtime doesn't contradict that the type of a variable is determined at compile time. > Of course you're right that Python won't stop you assigning (say) a float > to x. On the other hand, Pascal and C won't allow you to assign a string > to something which previously was an integer. The need to create new > labels for things just to placate the compiler is one of the more > annoying things about static typing. Yes, x was a number, now its a > string. If *I* can keep track of that, why can't the compiler? Is it really that annoying? Most examples I have seen here where the same variable was assigned multiple types, was labeled a code smell that needed to be approved by review or something like that, on this list. People praise the dynamic nature of Python here on this list and then often enough seem to recoil when they see a piece of code really using that dynamism. From antoon.pardon at vub.be Tue Feb 20 08:53:18 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 20 Feb 2018 14:53:18 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> >>>> I'm not proficient with C++, but IIUC, you could make a class in C++ and >>>> have the constructor and >>>> copy operator check for these kind of things. True it would be run-time >>>> checks but that would >>>> already be more than Python can give. >>> That (run-time checks) is exactly the same as Python gives. >> No it isn't exactly the same. Those runtime checks are implicit. They are like >> a contract made at declaration time, which make it impossible to assign something >> to the variable that doesn't meet the constraint without the program throwing an >> error. >> >> You can't do something like that in Python. > I *am* (reasonably) proficient in C++, and (other than the "values > have types rather than variables/names" point I already stated was a > difference in how the languages work) I don't see anything in what > you're describing as "things C++ can do" that Python can't. In C++ I can do something like: SomeClass MyVar; And after that the kind of possible assignments to MyVar are constraint. It makes the runtime throw an error when somewhere the program tries to assign something to MyVar that isn't allowed by SomeClass. You can't put such constraints on names in Python. In C++ I can do some like: ??? Some_Class: MyVar; And after that, It will be impossible to assign a value to MyVar that doesn't meet the constraints imposed by the constructor/copy operator. You have put somekind of contract on the name MyVar, that limits the kind of things assignable to it. You can't put such constraints on a name in Python. > > Sigh. Languages are different. That's my point. So, if languages are different, why the difficulty in accepting one can do something the other can't? > What's yours? If it's > that Python is worse than (some other language) then so what? Probably > true in some cases, but what makes you think you'll get enthusiastic > approval for such a statement in a Python group? So you only want to see praisal for python? One can't express how one prefers specific features from other languages? From rosuav at gmail.com Tue Feb 20 08:55:20 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Feb 2018 00:55:20 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On Wed, Feb 21, 2018 at 12:38 AM, Antoon Pardon wrote: > Why should this be done at compile time? I say a static language can do > the same as a dynamic language and your counter point is to ask for how > that static language can do something extra. > > The point I am making is that you claim dynamic languages are excelling > by being less demaning of the dynamic language. > > So yes the static language can check those kind of restraints at runtime > just as easily as a dynamic language. So what's the point of a "static language" if all it's doing is the same thing that any other language can do? The whole point of static analysis is that you can verify correctness BEFORE something gets into production. That means you have the potential to catch bugs before it's too late. If all you're going to do is check everything at run time, it's not static analysis any more. The question is: Can you create a strong compile-time type system that can verify correctness WITHOUT running the code? ChrisA From rosuav at gmail.com Tue Feb 20 08:58:03 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Feb 2018 00:58:03 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> Message-ID: On Wed, Feb 21, 2018 at 12:53 AM, Antoon Pardon wrote: > In C++ I can do something like: > > SomeClass MyVar; > > And after that the kind of possible assignments to MyVar are constraint. It > makes the runtime throw an error when somewhere the program tries to assign > something to MyVar that isn't allowed by SomeClass. > > You can't put such constraints on names in Python. > > In C++ I can do some like: > Some_Class: MyVar; > > And after that, It will be impossible to assign a value to MyVar that > doesn't meet the > constraints imposed by the constructor/copy operator. You have put > somekind of > contract on the name MyVar, that limits the kind of things assignable to > it. You > can't put such constraints on a name in Python. Okay. Now create a constraint on a name in C++ such that it can only accept integers representing A.D. years which, on the Gregorian calendar, are leap years. (Using a dedicated integer-like type is permitted.) It must accept all multiples of four, except those which are multiples of one hundred, unless they're also multiples of four hundred. That's what Steve asked for. Can you do it? Or is the C++ type system not flexible enough for that? ChrisA From bc at freeuk.com Tue Feb 20 09:03:34 2018 From: bc at freeuk.com (bartc) Date: Tue, 20 Feb 2018 14:03:34 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 20/02/2018 13:38, Antoon Pardon wrote: > People praise the dynamic nature of Python here on this list and then > often enough seem to recoil when they see a piece of code really using > that dynamism. Maybe everyone has their own ideas of how dynamic a language should be. (I use another language that I call 'dynamic'. But the only dynamic thing is that variables have a dynamic type - the variable's type is a runtime attribute. But, function names are static (they will always be function names). Module names are static. Type names (including user-defined ones) are static. Named constants are static. Source code is static (it only exists at compile-time). Attribute names are static; they have to be declared at compile-time. Variable names themselves are static: they can't become function or class or module names. Module imports are static (they are not done at runtime and can't be conditional). Operators are static and cannot overridden. AFAIK, all these are dynamic in Python (not sure about operators).) -- bartc From p.f.moore at gmail.com Tue Feb 20 09:04:03 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 20 Feb 2018 14:04:03 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> Message-ID: On 20 February 2018 at 13:53, Antoon Pardon wrote: > You can't put such constraints on names in Python. I know. That's what *I* said some time ago. >> Sigh. Languages are different. That's my point. > > So, if languages are different, why the difficulty in accepting one can > do something the other can't? I don't know. Who finds it difficult? I don't. What I find difficult is understanding what you're saying other languages can do that I haven't already agreed with. >> What's yours? If it's >> that Python is worse than (some other language) then so what? Probably >> true in some cases, but what makes you think you'll get enthusiastic >> approval for such a statement in a Python group? > > So you only want to see praisal for python? One can't express how one > prefers specific features from other languages? Of course you can. Go right ahead. Paul From ben.usenet at bsb.me.uk Tue Feb 20 09:10:07 2018 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 20 Feb 2018 14:10:07 +0000 Subject: could use some help with this problem! I've been working on it for days but cant seem to get it right ! References: <8a1de4c4-915a-4147-89f1-faca7ef62b0c@googlegroups.com> Message-ID: <87371v4urk.fsf@bsb.me.uk> Marc Cohen writes: > USING PYTHON 2: Why is that? > Write a program to play this game. This may seem tricky, so break it > down into parts. Like many programs, we have to use nested loops (one > loop inside another). In the outermost loop, we want to keep playing > until we are out of stones. You almost never /have/ to use nested loops. Has the course got this far without introducing the idea of a function? > So, the basic outline of the program should be something like this: > > totalStones = 100 > > maxStones = 5 maxTake or maxMove might be a more helpful name. > pile = TOTAL # all stones are in the pile to start > > while [pile is not empty]: > > while [player 1's answer is not valid]: > > [ask player 1] > > [execute player1?s move] > > Do the same for player 2?. (this can be achieved by a for loop) Is the idea for the program to play an optimal strategy for player 2, or is the program simply doing the housekeeping -- verifying moves and tracing the pile of stones? -- Ben. From antoon.pardon at vub.be Tue Feb 20 09:11:06 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 20 Feb 2018 15:11:06 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: <38bb9507-8676-a327-2d90-c025fc56681c@vub.be> On 20-02-18 14:55, Chris Angelico wrote: > On Wed, Feb 21, 2018 at 12:38 AM, Antoon Pardon wrote: >> Why should this be done at compile time? I say a static language can do >> the same as a dynamic language and your counter point is to ask for how >> that static language can do something extra. >> >> The point I am making is that you claim dynamic languages are excelling >> by being less demaning of the dynamic language. >> >> So yes the static language can check those kind of restraints at runtime >> just as easily as a dynamic language. > So what's the point of a "static language" if all it's doing is the > same thing that any other language can do? Who says that is all static languages are doing? Steven claimed dynamic languages excelled at runtime checks. I counter by stating that statically typed languages are just as good at runtime checks. How do you infer from that, that all that statically languages do, is the same as a dynamic language? -- Antoon From antoon.pardon at vub.be Tue Feb 20 09:13:33 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 20 Feb 2018 15:13:33 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> Message-ID: <28d70a52-81e4-1b04-0f71-60bc3a3d1d15@vub.be> On 20-02-18 15:04, Paul Moore wrote: > On 20 February 2018 at 13:53, Antoon Pardon wrote: > >> You can't put such constraints on names in Python. > I know. That's what *I* said some time ago. So why did you bother complaining to me when I wrote it? From antoon.pardon at vub.be Tue Feb 20 09:23:44 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 20 Feb 2018 15:23:44 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> Message-ID: <0ceaa666-a4c2-0562-d02d-8f85baa24ba9@vub.be> On 20-02-18 14:58, Chris Angelico wrote: > On Wed, Feb 21, 2018 at 12:53 AM, Antoon Pardon wrote: >> In C++ I can do something like: >> >> SomeClass MyVar; >> >> And after that the kind of possible assignments to MyVar are constraint. It >> makes the runtime throw an error when somewhere the program tries to assign >> something to MyVar that isn't allowed by SomeClass. >> >> You can't put such constraints on names in Python. >> >> In C++ I can do some like: >> Some_Class: MyVar; >> >> And after that, It will be impossible to assign a value to MyVar that >> doesn't meet the >> constraints imposed by the constructor/copy operator. You have put >> somekind of >> contract on the name MyVar, that limits the kind of things assignable to >> it. You >> can't put such constraints on a name in Python. > Okay. Now create a constraint on a name in C++ such that it can only > accept integers representing A.D. years which, on the Gregorian > calendar, are leap years. (Using a dedicated integer-like type is > permitted.) It must accept all multiples of four, except those which > are multiples of one hundred, unless they're also multiples of four > hundred. > > That's what Steve asked for. Can you do it? Or is the C++ type system > not flexible enough for that? Steve had multiple contributions in this thread. I didn't react to the one where he asked for that. I reacted to his assertion that dynamic languages excell at [run time checks] as if static languages are somehow limited at run time checks. writing a Digit Class in a dynamic language that checks whether the number you are about to assign is a digit, doesn't strike me as that much different from just writing a function that does the same kind of checking. Plus, in a statically typed language you can put constraints on the variables that will ensure the compilor will implicitly call the needed functions to check (because they are part of the constructor/copy operator) which you can't do in a dynamically typed language. So IMO the statically typed languages are at the advantage here. -- Antoon. From steve+comp.lang.python at pearwood.info Tue Feb 20 10:25:03 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 15:25:03 +0000 (UTC) Subject: Replying on a tweet with Twython References: <87po546jb1.fsf@munus.decebal.nl> <87lgfr7ryz.fsf@munus.decebal.nl> <87371z7453.fsf@munus.decebal.nl> Message-ID: On Sat, 17 Feb 2018 21:27:52 +0100, Cecil Westerhof wrote: [...] >> Okay. What happens when you try? > > Then the tweet is posted as a reply on the the first tweet. > > (So problem solved.) Ah, sorry, I did not understand your second post. I thought you were posting it as an update to the problem, and didn't understand that you were saying it was the solution to the problem. -- Steve From brozi at toya.net.pl Tue Feb 20 10:37:56 2018 From: brozi at toya.net.pl (AB) Date: Tue, 20 Feb 2018 16:37:56 +0100 Subject: solve_ivp problem (scipy 1.0.0) In-Reply-To: References: <2175be55-859c-23b9-4189-1c9a9e28729f@toya.net.pl> Message-ID: Hello I don't know why but my answer got lost somehow, so I'm sending it once more, hopefully with better luck. W dniu 2018-02-19 o?11:02, Thomas Jollans pisze: > On 2018-02-18 14:39, A.Brozi wrote: >> Hello >> >> In new "scipy" (1.0.0) I've found procedure "solve_ivp", which makes it >> possible to use "events" in order to terminate the ode solution when >> some condition is satisfied. >> The precise moment of termination (the value of independent variable) is >> to be found in "t_events", but I cannot find the dependent variable(s) >> value(s) at the termination moment. >> Am I missing something? Or it's simply not possible (hopefully 'not yet')? >> >> Regards >> Andrzej Brozi > > Isn't the solution passed back as ?y?? Yes, it is. But the vector "y" contains only values corresponding to the time moments given in "t_eval". It doesn't contain the value corresponding to the time of event function passing zero. In this situation a trajectory of a projectile hitting ground will terminate at a point above ground. I've succeeded to find one walk-around for this problem: if the "t_eval" vector is not supplied, then the last element of the solution "y" will be the desired value. Using this method it may be necessary to supply some reasonably small value of "max_step" (otherwise the solution may contain too few points). Unfortunately I don't see a simple way to guarantee the solution not to contain too many points (without "t_eval" it will contain all the points computed). Regards Andrzej Brozi From steve+comp.lang.python at pearwood.info Tue Feb 20 10:38:09 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 15:38:09 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> <0ceaa666-a4c2-0562-d02d-8f85baa24ba9@vub.be> Message-ID: On Tue, 20 Feb 2018 15:23:44 +0100, Antoon Pardon wrote: >> Okay. Now create a constraint on a name in C++ such that it can only >> accept integers representing A.D. years which, on the Gregorian >> calendar, are leap years. (Using a dedicated integer-like type is >> permitted.) It must accept all multiples of four, except those which >> are multiples of one hundred, unless they're also multiples of four >> hundred. >> >> That's what Steve asked for. Can you do it? Or is the C++ type system >> not flexible enough for that? > > Steve had multiple contributions in this thread. I didn't react to the > one where he asked for that. Yes you did: you refused to meet the challenge, stating (and I quote): "Why should this be done at compile time?" https://mail.python.org/pipermail/python-list/2018-February/730995.html -- Steve From chololennon at hotmail.com Tue Feb 20 10:43:35 2018 From: chololennon at hotmail.com (Cholo Lennon) Date: Tue, 20 Feb 2018 12:43:35 -0300 Subject: Python on Android? References: <20180218175734.6e8908d9@orbit.fritz.box> <874lmbbtq9.fsf@nightsong.com> Message-ID: On 20/02/18 11:50, Paul Rubin wrote: > Chris Angelico writes: >> probably means I'm going to have to learn Kivy > > Is SL4A still around? Maybe it's an alternative. > QPython 2 & 3, both have samples that use SL4A. -- Cholo Lennon Bs.As. ARG From chololennon at hotmail.com Tue Feb 20 11:01:33 2018 From: chololennon at hotmail.com (Cholo Lennon) Date: Tue, 20 Feb 2018 13:01:33 -0300 Subject: Python on Android? References: Message-ID: On 18/02/18 06:57, Chris Angelico wrote: > Does anyone have experience with running Python scripts on Android > phones? I have a brother (honestly! I'm not actually using a phone > myself!) who's trying to run one of my scripts in QPython, which > claims to be version 3.2.2. I think that really truly is a Python 3.2 > implementation - probing for newer features suggests that it actually > doesn't even support the u"..." syntax that came (back) in with Python > 3.3. So... does anyone know of a Python interpreter that's compatible > with 3.4 or better and runs on Android? > > Personal experiences trump just searching the web... > Currently people from QPython are implementing Python 3.6. They already have a beta version. On google play is called "QPy3.6 - Python3.6 for QPython (BETA)". Disclaimer: I didn't test it. -- Cholo Lennon Bs.As. ARG From steve+comp.lang.python at pearwood.info Tue Feb 20 11:44:44 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 16:44:44 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <85zi48b6r5.fsf@benfinney.id.au> <20180218125559.64d31761@wegge.dk> <20180219042632.79afa5b3@wegge.dk> <20180219140636.1dd40136@wegge.dk> <20180219163429.048ba054@wegge.dk> <20180220141208.6d81b8f0@wegge.dk> Message-ID: On Tue, 20 Feb 2018 14:12:08 +0100, Anders Wegge Keller wrote: > P? Tue, 20 Feb 2018 12:28:25 +0000 (UTC) Steven D'Aprano > skrev: >> On Mon, 19 Feb 2018 16:34:29 +0100, Anders Wegge Keller wrote: >> >> > P? Mon, 19 Feb 2018 15:15:19 +0000 (UTC) Steven D'Aprano >> > skrev: >> >> On Mon, 19 Feb 2018 14:06:36 +0100, Anders Wegge Keller wrote: >> >> >> [...] >> >> >> >> That's a mighty powerful claim that goes against the documentation >> >> for the array module. Can you back your claims up? >> >> >> >> Here's an array and a list: >> > >> > Make me an array of tuples with two integers and a string, and we >> > can talk. >> >> The array module's failure to support the specific type you want has no >> bearing on whether or not Python is statically typed. > > You claimed Array could do what I requested, i.e. container type that > guaranteed only one type inside. And that's exactly what array does: it guarantees only one type inside. It doesn't happen to be the type you want (a tuple of two integers and a string), but that's irrelevant to the question of whether the arrays and their content are strongly typed or not. If you want to say that array doesn't solve your problem, I have no argument with that. I'm not surprised: array is intentionally a very restricted type. I mentioned it as an example of something *similar* to what you want: a homogeneous sequence type with strong typing. I never said it would solve your (unknown, unstated) problems. https://mail.python.org/pipermail/python-list/2018-February/730916.html > Furthermore, you are misrepresenting C with malice. Am I? With *malice* you say. What gives you such insight into my state of mind? I don't think I've said very much about C except to say that it's known to not be type-safe. I don't think that ought to be controversial to anyone. C and C++ are well-known to be unsafe languages: https://blog.regehr.org/archives/213 and type-safety is a subset of that. That's why there's so much interest in new languages like Go and Rust. But I also suggested that some level of type-unsafety is probably unavoidable for a systems language. But okay, if I've misrepresented something about C, please explain, I'm happy to learn better. > I don't care why you feel that a simple observation is a personal > attack, Who mentioned anything about a personal attack? You're making technical claims that Python values aren't strongly typed. I think you're wrong, and said so, but offered to admit that *I* was wrong if you can demonstrate the correctness of your claims by actually changing the type of a list or array object in Python. You've been unable to do so. Rather than engage in good faith discussion, you're now trying to distract by making insinuations about my character (claiming I'm acting out of "malice", making me out to be complaining about "personal attacks") and now personal insults: > but I see you behave as a spoiled kid. > go and pout in your corner. The bottom line is, you've made technical claims that you are unable to support. If you can back up those claims, then I will learn something. How about you? -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 20 11:50:07 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 16:50:07 +0000 (UTC) Subject: Any users of statistics.mode() here? Message-ID: statistics.mode() currently raises an exception if there is more than one mode. Is anyone using this function in production? Or not using it because it doesn't do what you want, as the case may be? -- Steve From mwild at sun.ac.za Tue Feb 20 12:11:05 2018 From: mwild at sun.ac.za (Wild, Marcel, Prof ) Date: Tue, 20 Feb 2018 17:11:05 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <7wziC.125009$BM3.115116@fx14.am4> References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: I scarcely know Python, and I have no intention delving into it further. I was forced to use Python because it features binary decision diagrams, which MATHEMATICA doesn't. Coming from Mathematica the account of Nathan Murphy reads like a nightmare. The one point that stroke me the most was the schism between Python 2 and 3. No such thing with Mathematica: All its 11 or more versions are fully compatible, I never experienced any problems in this regard. Another point is the bad online help provided to "teach yourself" Python. For instance, it took me more than an hour to find out how to negate a Boolean variable, whereas in Mathematica you would just type "Negation" in the Wolfram Documentation search window, and get the information you need. I know one pays for Mathematica whereas Python is open source, but I've come to realize now that this money is very well spent! Question: Apart from a few commands not available in Mathematica, such as expr2bdd, is there really any domain of computation where Mathematica is inferior to Python? Marcel -----Original Message----- From: Python-list [mailto:python-list-bounces+mwild=sun.ac.za at python.org] On Behalf Of bartc Sent: 19 February 2018 02:35 PM To: python-list at python.org Subject: Re: Are the critiques in "All the things I hate about Python" valid? On 19/02/2018 02:59, Chris Angelico wrote: > On Mon, Feb 19, 2018 at 1:14 PM, bartc wrote: >> How would even a type for the odd numbers from 1 to 10 inclusive work? >> (That, a type consisting of one of the values in {1,3,5,7,9}.) Would >> they be ordered or unordered? Can I do arithmetic with them: will 3*3 >> work, but not 3*5? > > The type is "positive odd number below ten" and could be written as > int(1..9|1%2). That is an orderable type; you can say that 3 < 7, for > instance. And yes, arithmetic would be defined just fine; Sometimes, the reason for creating a special numerical type is precisely so you can't do arithmetic on them, if it's not meaningful for the type. So the special type of the values 65..90 might not allow the type be multiplied or divided, or added to itself. Because they represent characters A..Z. Or house numbers. Or the age of pensioners. (You'd need to convert to ordinary integers, is that is allowed.) there's no > requirement for the result of an operation to have the same type as > its inputs: > >>>> 5 / 2 # two integers > 2.5 Try that when the type of {1..13} represents playing card ordinal values. Type systems get rapidly very complicated when you have to deal with arbitrary sets of values and with arbitrary rules of interaction. Someone has to devise a programming language to allow all that without tying itself up in knots. Someone else has to program in it. And someone else has to try and understand it! Ones like C++ has already tied itself itself up in knots just doing the basics; I'm not sure how it would handle even my 1,3,5,7,9 type. But Python has classes and can do some of this stuff; how would it handle a numeric type that is constrained to be whole numbers within 0..9 inclusive? -- bartc -- https://mail.python.org/mailman/listinfo/python-list The integrity and confidentiality of this email is governed by these terms / Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende bepalings gere?l. http://www.sun.ac.za/emaildisclaimer From ian.g.kelly at gmail.com Tue Feb 20 12:17:12 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 20 Feb 2018 10:17:12 -0700 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> <0ceaa666-a4c2-0562-d02d-8f85baa24ba9@vub.be> Message-ID: On Tue, Feb 20, 2018 at 8:38 AM, Steven D'Aprano wrote: > On Tue, 20 Feb 2018 15:23:44 +0100, Antoon Pardon wrote: > >>> Okay. Now create a constraint on a name in C++ such that it can only >>> accept integers representing A.D. years which, on the Gregorian >>> calendar, are leap years. (Using a dedicated integer-like type is >>> permitted.) It must accept all multiples of four, except those which >>> are multiples of one hundred, unless they're also multiples of four >>> hundred. >>> >>> That's what Steve asked for. Can you do it? Or is the C++ type system >>> not flexible enough for that? >> >> Steve had multiple contributions in this thread. I didn't react to the >> one where he asked for that. > > Yes you did: you refused to meet the challenge, stating (and I quote): > > "Why should this be done at compile time?" > > https://mail.python.org/pipermail/python-list/2018-February/730995.html I really don't understand what point you're driving at here, Steven. The original claim (by you) was that dynamic languages excel at "enforcing constraints at run-time which are hard to enforce at compile-time". The counter-argument, as I understand, it was that while the constraints may be hard to enforce at compile-time, they are just as easy to enforce at run-time in a static language as in a dynamic language, so that can't really be considered a *strength* per se of dynamic languages. You then followed this up by issuing a challenge to enforce this as a compile-type check in C++. Obviously this has not been done [1], but just as obviously it could not be done at compile-time in a dynamic language either, so I don't see how any of this justifies your claim that being a static language somehow makes constraints harder to enforce. [1] But I have no doubt that it could be done in a language with a sufficiently advanced type system. Just by way of example, the Haskell wiki offers an example of quicksort implemented in the Haskell type system: https://wiki.haskell.org/Type_arithmetic#An_Advanced_Example_:_Type-Level_Quicksort From rantingrickjohnson at gmail.com Tue Feb 20 13:02:02 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 20 Feb 2018 10:02:02 -0800 (PST) Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> <0ceaa666-a4c2-0562-d02d-8f85baa24ba9@vub.be> Message-ID: <3620dcfc-6679-4107-83b0-d42f00e332e1@googlegroups.com> On Tuesday, February 20, 2018 at 9:40:37 AM UTC-6, Steven D'Aprano wrote: [...] > Yes you did: you refused to meet the challenge, stating (and I quote): I'm always entertained by Steven's so-called "challenges". You see, Steven is addicted to winning, and he'll do anything to win a debate, even construct a challenge that is obviously untenable. A typical example of Steven's absurd challenges goes something like this: from absurd.challenges.missonimpossible import Message body = """ MISSON: MOVE YOURSELF FROM YOUR CURRENT POSITION ON THE ON THIS EARTH TO THE TOP OF THE EIFFEL TOWER IN PARIS, FRANCE. CAVEAT_1: You cannot use public or private modes of transportation. Including, (but not limited to): planes, trains automobiles, boats, jet-skis, roller- skates, skate boards, kites, gliders, bicycles, tricycles, unicycles, big- wheels, hot-air balloons, magic carpets, etc... CAVEAT_2: Neither may you move yourself using your own forms of bodily locomotion. Including (but not limited to): crawling, walking, running, jogging, rolling, skipping, jumping, hopping, cartwheels, backflips, etc... CAVEAT_3: Neither may you utilize the animal kingdom to transport yourself. Including (but not limited to): Being carried by an army of ants, a flock of birds, a gang of gorilla, or a pod of orca, etc... CAVEAT_4: If you happen to be at the top of the Eiffel tower upon learning of this challenge, it doesn't count. But please do post a pic on Instagram, as i rather enjoy a good view. Thanks :-) CAVEAT_5: Teleportation is strictly forbidden! Although it'd be a neat trick. And if you have a working prototype i'd love to stop by for a personal demonstration and have a chance to make copies of the schematics and the code base. (for documentation purposes only, of course) This is your misson, should you choose to accept it... """ if __name__ == '__main__': msg = Message(title="Loco-motion", body=body) msg.play_async("misson_impossible.mp3") print msg msg.after(10, exit) raw_input('\nThis message will self destruct in ~10 seconds...') From Richard at Damon-family.org Tue Feb 20 13:05:34 2018 From: Richard at Damon-family.org (Richard Damon) Date: Tue, 20 Feb 2018 13:05:34 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> Message-ID: <20C59850-0476-48C2-94CD-5161B9158C7B@Damon-family.org> > On Feb 20, 2018, at 8:58 AM, Chris Angelico wrote: > >> On Wed, Feb 21, 2018 at 12:53 AM, Antoon Pardon wrote: >> In C++ I can do something like: >> >> SomeClass MyVar; >> >> And after that the kind of possible assignments to MyVar are constraint. It >> makes the runtime throw an error when somewhere the program tries to assign >> something to MyVar that isn't allowed by SomeClass. >> >> You can't put such constraints on names in Python. >> >> In C++ I can do some like: >> Some_Class: MyVar; >> >> And after that, It will be impossible to assign a value to MyVar that >> doesn't meet the >> constraints imposed by the constructor/copy operator. You have put >> somekind of >> contract on the name MyVar, that limits the kind of things assignable to >> it. You >> can't put such constraints on a name in Python. > > Okay. Now create a constraint on a name in C++ such that it can only > accept integers representing A.D. years which, on the Gregorian > calendar, are leap years. (Using a dedicated integer-like type is > permitted.) It must accept all multiples of four, except those which > are multiples of one hundred, unless they're also multiples of four > hundred. > > That's what Steve asked for. Can you do it? Or is the C++ type system > not flexible enough for that? > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list Such a class would be fairly trivial to write in C++, or I suspect Python. In C++ you would have a constructor and possibilities an assignment operator that takes an integer, tests it for validity and throw/asserts if it is incorrect. (Depending on other requirements, you might allow or not implicit conversions) It probably also has a constructor and an assignment operator that takes a ?LeapYear? and just uses it. To be usable, it will need something to allow you to get the integer value out, might be implicit or explicit. The big issue with such a type, and why it doesn?t make much sense as a type, is that there is very little that can be done with such a type, as there is no interesting operation for which the type is at least mostly closed under, at best it is closed under +400*n From rhodri at kynesim.co.uk Tue Feb 20 13:42:41 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 20 Feb 2018 18:42:41 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: On 20/02/18 17:11, Wild, Marcel, Prof wrote: > I scarcely know Python, and I have no intention delving into it further. > I was forced to use Python because it features binary decision diagrams, which MATHEMATICA doesn't. Coming from Mathematica the account of Nathan Murphy reads like a nightmare. > I have to admit, I have no idea what you're talking about. I suspect, though, that you would find a similar discussion of any other programming language at least as nightmarish. > The one point that stroke me the most was the schism between Python 2 and 3. No such thing with Mathematica: All its 11 or more versions are fully compatible, I never experienced any problems in this regard. The schism is not as wide as you are implying. Aside from "print" becoming a function, which is blindingly obvious whenever you trip over it, there is relatively little reason why an ordinary Pythonista would care whether he or she was running Python 2 or Python 3. Python 3.5 vs Python 3.7 is much more likely to be a relevant question, because of course Python has evolved new features over time. The statement "all its [...] versions are fully compatible" implies Mathematica hasn't evolved over those versions. I sincerely doubt that is true. > Another point is the bad online help provided to "teach yourself" Python. For instance, it took me more than an hour to find out how to negate a Boolean variable, whereas in Mathematica you would just type "Negation" in the Wolfram Documentation search window, and get the information you need. This is likely to be a personal thing. Mathematica ties you firmly to its IDE; you get all the bells and whistles of that IDE, but only the bells and whistles of that IDE. Python doesn't tie you to anything in particular, so you have to provide your own bells and whistles (but can provide any you can find or create). That said, you are not making a good case for your research skills. Googling "python boolean negation" got me the information in under a minute, including the Firefox startup time. Reading through the Boolean Expressions part of the online documentation at docs.python.com took little longer, though admittedly that isn't meant for beginners. Even firing up a Python interpreter and typing >>> help("not") didn't take that long (and honestly, "negation" is not the first word I think of when inverting booleans). > > I know one pays for Mathematica whereas Python is open source, but I've come to realize now that this money is very well spent! > > Question: Apart from a few commands not available in Mathematica, such as expr2bdd, is there really any domain of computation where Mathematica is inferior to Python? Not knowing much about Mathematica, all I can say is "almost certainly." -- Rhodri James *-* Kynesim Ltd From ben+python at benfinney.id.au Tue Feb 20 13:45:10 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 21 Feb 2018 05:45:10 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: <85h8qbbivd.fsf@benfinney.id.au> "Wild, Marcel, Prof " writes: > I scarcely know Python, and I have no intention delving into it > further. That's fine. This is a forum for those who do have an ongoing interest in Python. I think your needs would be better served elsewhere. -- \ ?Prediction is very difficult, especially of the future.? | `\ ?Niels Bohr | _o__) | Ben Finney From bc at freeuk.com Tue Feb 20 14:17:08 2018 From: bc at freeuk.com (bartc) Date: Tue, 20 Feb 2018 19:17:08 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> Message-ID: On 20/02/2018 19:04, Dennis Lee Bieber wrote: > On Tue, 20 Feb 2018 17:11:05 +0000, "Wild, Marcel, Prof " > declaimed the following: > >> >> So the special type of the values 65..90 might not allow the type be multiplied or divided, or added to itself. Because they represent characters A..Z. Or house numbers. Or the age of pensioners. (You'd need to convert to ordinary integers, is that is allowed.) > > > Off-hand -- if you are storing the /age of pensioners/, you have an > inappropriate data model... Age being a time varying value computed as: > time_now - date_of_birth > and date_of_birth is the proper entity for storage... If you wanted a scientifically exact value, maybe. But someone who's 24.157094 years old now won't say their age is 24.157094 (and 24.157104 five minutes later). They will usually say they are 24, until they are 25 (much older people prefer to round upwards, for some reason). So age is usually colloquially specified as an integer from 1 to around 100. Other than for young children where the lack of precision requires the use of fractions or to switch to whole numbers of months or weeks. -- bartc From grant.b.edwards at gmail.com Tue Feb 20 14:25:07 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 20 Feb 2018 19:25:07 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: On 2018-02-20, Rhodri James wrote: > The schism is not as wide as you are implying. Aside from "print" > becoming a function, which is blindingly obvious whenever you trip over > it, there is relatively little reason why an ordinary Pythonista would > care whether he or she was running Python 2 or Python 3. Any ordinary Pythonista to deals with raw data "bytes" cares a great deal. There are major differences between the Py2 and Py3 in that area, and they're a royal PITA to deal with. -- Grant Edwards grant.b.edwards Yow! Are you mentally here at at Pizza Hut?? gmail.com From geld at sun.ac.za Tue Feb 20 14:39:33 2018 From: geld at sun.ac.za (Geldenhuys, J, Prof ) Date: Tue, 20 Feb 2018 19:39:33 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: <9E15048A-2265-42F3-868B-98B138747F39@sun.ac.za> Hi Marcel, I have read through the article and it seems to me that all of the problems described boil down to one important point: use the right tool for the right job. Python is probably not the best language for developing large, complex systems (although there are such systems). Most (or all?) of the concerns that Murthy raises (dynamic typing, support for multithreading, performance, Python2/3, encapsulation) speak to that. He is not wrong about any of the points, but it could be said that he is expecting too much. We don't expect a carpenter with a hammer to build an entire apartment building, and we don't expect a big construction company to build a coffee table. Python does have many desirable features: it is easy to learn and the code is relatively clear. It is available on most platforms (I can run it directly on my phone), and it is easy to port to new platforms. There are many things that Mathematica can do that Python can't, and vice versa. Right tool/right job. I have often used Python to build prototypes to see if an algorithm works, but then I use a more efficient language (like Java or C) for the "real" implementation. Out of interest, Mathematica itself is written in C, C++, and Java, and consists of a few million lines of code. Python has several implementations; the "reference" implementation is CPython which is written in C and consists of ~540,000 lines of code. Because Python is open-source, it has the strengths (many contributors) and weaknesses (more difficult to coordinate development) that comes from this paradigm. As far as the documentation is concerned, Python is considered a well-documented language. Of course, individual libraries may not be, but as it happens, pyeda has good documentation (http://pyeda.readthedocs.io/en/latest/). I think you may just have been unlucky not to have found it. I think your case illustrates the Python/Mathematica issue well: you found a job for which Mathematica was not the perfect tool and you used Python. At the end of the day, both M & P have their place. For example, we probably won't use either to teach Introduction Computer Science soon, because they both lack features that we expect our students to be familiar with at the end of their first year. Prof Jaco Geldenhuys Medeprofessor: Rekenaarwetenskap | Associate Professor: Computer Science Fakulteit Natuurwetenskappe | Faculty of Science e: geld at sun.ac.za | t: +27 21 808 4232 | a: Algemene Ingenieurswese Gebou A519 | General Engineering Building A519 ?On 2018/02/20, 19:11, "Wild, Marcel, Prof " wrote: I scarcely know Python, and I have no intention delving into it further. I was forced to use Python because it features binary decision diagrams, which MATHEMATICA doesn't. Coming from Mathematica the account of Nathan Murphy reads like a nightmare. The one point that stroke me the most was the schism between Python 2 and 3. No such thing with Mathematica: All its 11 or more versions are fully compatible, I never experienced any problems in this regard. Another point is the bad online help provided to "teach yourself" Python. For instance, it took me more than an hour to find out how to negate a Boolean variable, whereas in Mathematica you would just type "Negation" in the Wolfram Documentation search window, and get the information you need. I know one pays for Mathematica whereas Python is open source, but I've come to realize now that this money is very well spent! Question: Apart from a few commands not available in Mathematica, such as expr2bdd, is there really any domain of computation where Mathematica is inferior to Python? Marcel -----Original Message----- From: Python-list [mailto:python-list-bounces+mwild=sun.ac.za at python.org] On Behalf Of bartc Sent: 19 February 2018 02:35 PM To: python-list at python.org Subject: Re: Are the critiques in "All the things I hate about Python" valid? On 19/02/2018 02:59, Chris Angelico wrote: > On Mon, Feb 19, 2018 at 1:14 PM, bartc wrote: >> How would even a type for the odd numbers from 1 to 10 inclusive work? >> (That, a type consisting of one of the values in {1,3,5,7,9}.) Would >> they be ordered or unordered? Can I do arithmetic with them: will 3*3 >> work, but not 3*5? > > The type is "positive odd number below ten" and could be written as > int(1..9|1%2). That is an orderable type; you can say that 3 < 7, for > instance. And yes, arithmetic would be defined just fine; Sometimes, the reason for creating a special numerical type is precisely so you can't do arithmetic on them, if it's not meaningful for the type. So the special type of the values 65..90 might not allow the type be multiplied or divided, or added to itself. Because they represent characters A..Z. Or house numbers. Or the age of pensioners. (You'd need to convert to ordinary integers, is that is allowed.) there's no > requirement for the result of an operation to have the same type as > its inputs: > >>>> 5 / 2 # two integers > 2.5 Try that when the type of {1..13} represents playing card ordinal values. Type systems get rapidly very complicated when you have to deal with arbitrary sets of values and with arbitrary rules of interaction. Someone has to devise a programming language to allow all that without tying itself up in knots. Someone else has to program in it. And someone else has to try and understand it! Ones like C++ has already tied itself itself up in knots just doing the basics; I'm not sure how it would handle even my 1,3,5,7,9 type. But Python has classes and can do some of this stuff; how would it handle a numeric type that is constrained to be whole numbers within 0..9 inclusive? -- bartc -- https://mail.python.org/mailman/listinfo/python-list The integrity and confidentiality of this email is governed by these terms / Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende bepalings gere?l. http://www.sun.ac.za/emaildisclaimer From bc at freeuk.com Tue Feb 20 14:46:37 2018 From: bc at freeuk.com (bartc) Date: Tue, 20 Feb 2018 19:46:37 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? (Posting On Python-List Prohibited) In-Reply-To: <2be24305-99bb-4c0a-b23b-caff91ca769d@googlegroups.com> References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2be24305-99bb-4c0a-b23b-caff91ca769d@googlegroups.com> Message-ID: On 20/02/2018 19:35, Lawrence D?Oliveiro wrote: > On Wednesday, February 21, 2018 at 1:43:41 AM UTC+13, bartc wrote: >> In Pascal (and presumably Ada) then all the >> gubbins need to make this work properly: >> >> var x: 1..10; >> >> x = 10; >> x = x + 1; { error? } > > Error on both statements. Pascal doesn?t allow a statement to just consist of an expression. At least, it didn?t the last time I checked. OK, I forgot it needs := for assignment. > Besides, it?s not clear what the point is of doing a comparison between those terms and throwing the result away. Many languages including Python allow exactly that. (The ones I create make it an error. Only certain kinds of expression terms also have versions that can meaningfully be independent statements.) -- bartc From python at mrabarnett.plus.com Tue Feb 20 15:17:55 2018 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 20 Feb 2018 20:17:55 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> Message-ID: <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> On 2018-02-20 19:17, bartc wrote: > On 20/02/2018 19:04, Dennis Lee Bieber wrote: >> On Tue, 20 Feb 2018 17:11:05 +0000, "Wild, Marcel, Prof " >> declaimed the following: >> >>> >>> So the special type of the values 65..90 might not allow the type be multiplied or divided, or added to itself. Because they represent characters A..Z. Or house numbers. Or the age of pensioners. (You'd need to convert to ordinary integers, is that is allowed.) >> >> >> Off-hand -- if you are storing the /age of pensioners/, you have an >> inappropriate data model... Age being a time varying value computed as: >> time_now - date_of_birth >> and date_of_birth is the proper entity for storage... > > If you wanted a scientifically exact value, maybe. > > But someone who's 24.157094 years old now won't say their age is > 24.157094 (and 24.157104 five minutes later). They will usually say they > are 24, until they are 25 (much older people prefer to round upwards, > for some reason). > > So age is usually colloquially specified as an integer from 1 to around > 100. Other than for young children where the lack of precision requires > the use of fractions or to switch to whole numbers of months or weeks. > The point he was making is that if you store a person's age, you'd have to update it every year. It's far better to store the date of birth and calculate the age on demand. From jsf80238 at gmail.com Tue Feb 20 15:20:14 2018 From: jsf80238 at gmail.com (Jason Friedman) Date: Tue, 20 Feb 2018 13:20:14 -0700 Subject: Any users of statistics.mode() here? In-Reply-To: References: Message-ID: > statistics.mode() currently raises an exception if there is more than one > mode. > I am an infrequent user of this package and this function. My two cents: * Leave the current behavior as-is. * Continue to throw an exception for no data. * Add an argument, named perhaps mutli=False, that if set to True causes the function to a return a list of modes, including if there is only a single mode. statistics.mode((1, 2, 2, 3, 3, 4, 4, 5), multi=True) would return [2, 3, 4]. * Would still throw StatisticsError if no data or if every element occurs exactly the same number of times (for example, statistics.mode(range(1000000000), multi=True)) From rantingrickjohnson at gmail.com Tue Feb 20 15:42:23 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 20 Feb 2018 12:42:23 -0800 (PST) Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Tuesday, February 20, 2018 at 2:18:31 PM UTC-6, MRAB wrote: > The point he was making is that if you store a person's age, you'd have > to update it every year. It's far better to store the date of birth and > calculate the age on demand. *AHEM* At the risk of being labeled a "quibbler" (which, in the grander scheme is not really all that bad considering some of the names that have been attributed to me), i must say your advice is only "sound advice" in cases where the age will not be queried frequently. For instance, if the age is queried many times a second, it would be a much wiser design to set-up an event that will advance the age at the end of the last second of every year, instead of doing the age calculation many times a second. Heck, even if the frequency is multiple time a day, a valid argument could be made. Of course. Outside of buggy loop that, much to the chagrin of the code monkey who wrote it waxes infinite, i cannot image many scenarios in which someone's age would be queried many times a second. But hey, this is Python-list after all. And i wouldn't be a true pal i didn't point out an implicit corner case. From bc at freeuk.com Tue Feb 20 15:47:44 2018 From: bc at freeuk.com (bartc) Date: Tue, 20 Feb 2018 20:47:44 +0000 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On 20/02/2018 20:17, MRAB wrote: > On 2018-02-20 19:17, bartc wrote: >> On 20/02/2018 19:04, Dennis Lee Bieber wrote: >>> On Tue, 20 Feb 2018 17:11:05 +0000, "Wild, Marcel, Prof >>> " >>> declaimed the following: >>> >>>> >>>> So the special type of the values 65..90 might not allow the type be >>>> multiplied or divided, or added to itself. Because they represent >>>> characters A..Z. Or house numbers. Or the age of pensioners. (You'd >>>> need to convert to ordinary integers, is that is allowed.) >>> >>> >>> ????Off-hand -- if you are storing the /age of pensioners/, you have an >>> inappropriate data model... Age being a time varying value computed as: >>> ??????? time_now - date_of_birth >>> and date_of_birth is the proper entity for storage... >> >> If you wanted a scientifically exact value, maybe. >> >> But someone who's 24.157094 years old now won't say their age is >> 24.157094 (and 24.157104 five minutes later). They will usually say they >> are 24, until they are 25 (much older people prefer to round upwards, >> for some reason). >> >> So age is usually colloquially specified as an integer from 1 to around >> 100. Other than for young children where the lack of precision requires >> the use of fractions or to switch to whole numbers of months or weeks. >> > The point he was making is that if you store a person's age, you'd have > to update it every year. It's far better to store the date of birth and > calculate the age on demand. People are making too much of my example of a type consisting of a set of special values. Anyway the values might not relate to an individual; they could be indices into a data structure so that C[65] gives you the number of 65-year-olds or something, or 67 is the age at which someone is entitled to a pension. -- bartc From rosuav at gmail.com Tue Feb 20 15:51:21 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Feb 2018 07:51:21 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Wed, Feb 21, 2018 at 7:42 AM, Rick Johnson wrote: > On Tuesday, February 20, 2018 at 2:18:31 PM UTC-6, MRAB wrote: > >> The point he was making is that if you store a person's age, you'd have >> to update it every year. It's far better to store the date of birth and >> calculate the age on demand. > > *AHEM* > > At the risk of being labeled a "quibbler" (which, in the > grander scheme is not really all that bad considering some > of the names that have been attributed to me), i must say > your advice is only "sound advice" in cases where the age > will not be queried frequently. > > For instance, if the age is queried many times a second, it > would be a much wiser design to set-up an event that will > advance the age at the end of the last second of every year, > instead of doing the age calculation many times a second. > Heck, even if the frequency is multiple time a day, a valid > argument could be made. Nope. Even if you need the age many times per second, it's still better to store the date of birth, because you eliminate boundary conditions and duplicated data. But that's assuming you're storing the age of *a person*. If the age in question is a boundary ("motor insurance premiums are increased if the driver is 65 years or older"), then it's still sane to store the age. ChrisA From python at mrabarnett.plus.com Tue Feb 20 15:51:58 2018 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 20 Feb 2018 20:51:58 +0000 Subject: Any users of statistics.mode() here? In-Reply-To: References: Message-ID: On 2018-02-20 20:20, Jason Friedman wrote: >> statistics.mode() currently raises an exception if there is more than one >> mode. >> > > I am an infrequent user of this package and this function. My two cents: > * Leave the current behavior as-is. > * Continue to throw an exception for no data. > * Add an argument, named perhaps mutli=False, that if set to True causes > the function to a return a list of modes, including if there is only a > single mode. statistics.mode((1, 2, 2, 3, 3, 4, 4, 5), multi=True) would > return [2, 3, 4]. > * Would still throw StatisticsError if no data or if every element occurs > exactly the same number of times (for example, > statistics.mode(range(1000000000), multi=True)) > I think that it would be better to have a separate function instead of having the function sometimes return a single value and sometimes a list of values. There's an interesting answer here: http://mathforum.org/library/drmath/view/61375.html From auriocus at gmx.de Tue Feb 20 16:19:13 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 20 Feb 2018 22:19:13 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> Message-ID: Am 20.02.18 um 14:58 schrieb Chris Angelico: > Okay. Now create a constraint on a name in C++ such that it can only > accept integers representing A.D. years which, on the Gregorian > calendar, are leap years. (Using a dedicated integer-like type is > permitted.) It must accept all multiples of four, except those which > are multiples of one hundred, unless they're also multiples of four > hundred. > > That's what Steve asked for. Can you do it? Or is the C++ type system > not flexible enough for that? It might be possible to do that with compile-time checking for compile-time constants. For runtime checking, you already got the answer - provide a class LeapYear { LeapYear(int y) { if (!condition....) throw std::runtime_eror(); } operator = (int y) { ...also test and throw... } }; then: LeapYear y2000 = 2000; // works y2000 = 2004; // works y2000 = 2001; // throws an error The difference to Python is that "a=b" and "a+=b" are similar in C++, in both cases you call a method on the object "a", whereas in Python the first one is handled independently of the type of "a". If you insist in compile-time checking, I think it's possible, though I'm not one of those C++ wizards to implement it from up my sleeve. The basic technique will be that you make your LeapYear constructible/assignable from a static type only where constructino of the type fails if the condition is not met. So class LeapYear { ... template operator = (LeapYearCheck y) { // at this point, we are sure that LeapYearCheck succeeded } }; LeapYearCheck could be implemented using template metaprogramming (quite horrible) or the new funky constexpr feature in C++11/C++14 (less horrible). However this will throw an error whenever the compiler can't prove that the year is a leap year, effectively only when it can be computed at compile time, which certainly limits the usefulness of this type. From rantingrickjohnson at gmail.com Tue Feb 20 17:01:50 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 20 Feb 2018 14:01:50 -0800 (PST) Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: <2f246c96-f4dc-4678-a78f-ec3d17ac9c0a@googlegroups.com> On Tuesday, February 20, 2018 at 2:51:56 PM UTC-6, Chris Angelico wrote: [...] > Nope. Even if you need the age many times per second, it's still > better to store the date of birth, because you eliminate boundary > conditions and duplicated data. You failed to provide any examples proving this assertion of yours. I can't imagine how either would be a problem. > But that's assuming you're storing the age of *a person*. > If the age in question is a boundary ("motor insurance > premiums are increased if the driver is 65 years or > older"), then it's still sane to store the age. The boundary problem you outline here is a simple matter of checking the age annually and increasing the premium if the person is 65 years or older, and has nothing to do with whether or not the age is calculated once a year or upon every request. def annual_probe(self, geriatricMarker=65): self.age += 1 if self.age >= geriatricMarker: self.increase_premium() But obviously this "data" should reside in a database, not be instanced into OO objects. From rosuav at gmail.com Tue Feb 20 17:10:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Feb 2018 09:10:08 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <2f246c96-f4dc-4678-a78f-ec3d17ac9c0a@googlegroups.com> References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> <2f246c96-f4dc-4678-a78f-ec3d17ac9c0a@googlegroups.com> Message-ID: On Wed, Feb 21, 2018 at 9:01 AM, Rick Johnson wrote: > On Tuesday, February 20, 2018 at 2:51:56 PM UTC-6, Chris Angelico wrote: > [...] >> Nope. Even if you need the age many times per second, it's still >> better to store the date of birth, because you eliminate boundary >> conditions and duplicated data. > > You failed to provide any examples proving this assertion of > yours. I can't imagine how either would be a problem. https://en.wikipedia.org/wiki/Database_normalization I'm not going to do your research for you. ChrisA From androidmaroso at gmail.com Tue Feb 20 17:41:42 2018 From: androidmaroso at gmail.com (Maroso Marco) Date: Tue, 20 Feb 2018 14:41:42 -0800 (PST) Subject: How to transform this as a service Message-ID: Hi everyone, i need this program to run as a service. This program basically check an email account (gmail) and looks if there is mail from a specific email account, then reads the subject line and splits the text found assigning the left part (before the ; ) to the password, and the right part to the comand to be executed. It should loop continuosly waiting a specific time given in a file called timing.ini (seconds) My program seems to have a problem and i don't understand which! Firstly it worked and now just went to run it again and it doesnt. I have different versions of python installed on my pc. Can someone please tell me how to transform it in a service to run on Windows? I found some solutions but i don't understand how to implement it correctly This is my code, any help is really apreciated, i'm loosing my nights on this and can't get it working. import imaplib import os import email import email.header import time import subprocess def mailcontroller(): # Set user, pass and allowed mail for giving commands plusmail = "anemailaddress at gmail.com" googlepass = "thepassword" captain = "autorizedemailaccount at gmail.com" # Set vars for IMAP access M = imaplib.IMAP4_SSL('imap.gmail.com') M.login(plusmail, googlepass) M.select() # Set search on UNSEEN messages status, response = M.search(None, '(UNSEEN)') unread_msg_nums = response[0].split() # Mark as read for e_id in unread_msg_nums: M.store(e_id, '+FLAGS', '\Seen') # cycle messages sent from autorized email address typ, data = M.search(None, 'From',(captain)) for num in data[0].split(): typ, data = M.fetch(num, '(RFC822)') msg = email.message_from_string(data[0][1]) decode = email.header.decode_header(msg['Subject'])[0] subject = unicode(decode[0]) comando = subject if googlepass in subject: # print 'Message %s: %s' % (num, subject) # Split subject line googlepass,comando = subject.split(";") # Execute command #os.system(comando) # Execute command with alternate method subprocess.call(comando) # Delete email M.store(num, '+FLAGS', '\\Deleted') M.close() M.logout() # Read ini file for timer settings timing = open('timing.ini', 'r').read() # Convert timer value from string to int time.sleep(int(timing)) while True: mailcontroller() From androidmaroso at gmail.com Tue Feb 20 17:43:01 2018 From: androidmaroso at gmail.com (Maroso Marco) Date: Tue, 20 Feb 2018 14:43:01 -0800 (PST) Subject: How to transform this as a service In-Reply-To: References: Message-ID: <26c250ef-0ee8-4361-b05f-0ef366e9800d@googlegroups.com> Il giorno marted? 20 febbraio 2018 23:42:27 UTC+1, Maroso Marco ha scritto: > Hi everyone, i need this program to run as a service. > > This program basically check an email account (gmail) and looks if there is > mail from a specific email account, then reads the subject line and splits the text found assigning the left part (before the ; ) to the password, and the right part to the comand to be executed. > It should loop continuosly waiting a specific time given in a file called timing.ini (seconds) > > My program seems to have a problem and i don't understand which! Firstly it worked and now just went to run it again and it doesnt. I have different versions of python installed on my pc. > > Can someone please tell me how to transform it in a service to run on Windows? > > I found some solutions but i don't understand how to implement it correctly > This is my code, any help is really apreciated, i'm loosing my nights on this and can't get it working. > > > > > import imaplib > import os > import email > import email.header > import time > import subprocess > > def mailcontroller(): > > # Set user, pass and allowed mail for giving commands > plusmail = "anemailaddress at gmail.com" > googlepass = "thepassword" > captain = "autorizedemailaccount at gmail.com" > > # Set vars for IMAP access > M = imaplib.IMAP4_SSL('imap.gmail.com') > M.login(plusmail, googlepass) > M.select() > > # Set search on UNSEEN messages > status, response = M.search(None, '(UNSEEN)') > unread_msg_nums = response[0].split() > > > # Mark as read > for e_id in unread_msg_nums: > M.store(e_id, '+FLAGS', '\Seen') > > # cycle messages sent from autorized email address > typ, data = M.search(None, 'From',(captain)) > > for num in data[0].split(): > typ, data = M.fetch(num, '(RFC822)') > msg = email.message_from_string(data[0][1]) > decode = email.header.decode_header(msg['Subject'])[0] > subject = unicode(decode[0]) > comando = subject > > if googlepass in subject: > > # print 'Message %s: %s' % (num, subject) > # Split subject line > googlepass,comando = subject.split(";") > # Execute command > #os.system(comando) > # Execute command with alternate method > subprocess.call(comando) > # Delete email > M.store(num, '+FLAGS', '\\Deleted') > > M.close() > M.logout() > > # Read ini file for timer settings > timing = open('timing.ini', 'r').read() > # Convert timer value from string to int > time.sleep(int(timing)) > > while True: > mailcontroller() I would like to implement my code in something like this described here : http://www.chrisumbel.com/article/windows_services_in_python From mailman at hanez.org Tue Feb 20 18:11:24 2018 From: mailman at hanez.org (Johannes Findeisen) Date: Wed, 21 Feb 2018 00:11:24 +0100 Subject: Python on Android? In-Reply-To: References: <20180218175734.6e8908d9@orbit.fritz.box> Message-ID: <20180221001124.16be9365@orbit.fritz.box> On Tue, 20 Feb 2018 23:12:23 +1100 Chris Angelico wrote: > On Mon, Feb 19, 2018 at 3:57 AM, Johannes Findeisen wrote: > > On Sun, 18 Feb 2018 20:57:02 +1100 > > Chris Angelico wrote: > > > >> Does anyone have experience with running Python scripts on Android > >> phones? I have a brother (honestly! I'm not actually using a phone > >> myself!) who's trying to run one of my scripts in QPython, which > >> claims to be version 3.2.2. I think that really truly is a Python 3.2 > >> implementation - probing for newer features suggests that it actually > >> doesn't even support the u"..." syntax that came (back) in with Python > >> 3.3. So... does anyone know of a Python interpreter that's compatible > >> with 3.4 or better and runs on Android? > >> > > > > There is an App for Android called "Pydroid 3". You can find it in > > the Google Play Store [0]. It provides a Python interpreter in version > > 3.6.2 in its current release. > > > > The Python binary is installed under > > > > /data/user/0/ru.iiec.pydroid3/files/arm-linux-androideabi/bin/python > > > > but I can not access it in a normal terminal without being user "root" > > but it is usable from within the terminal included in the Pydroid App. > > Many thanks for all the suggestions. Pydroid is the one that has > worked; he's now able to run stuff happily from the terminal. Next > step is to create an icon that invokes it, but that's looking rather > harder (probably means I'm going to have to learn Kivy, and I already > have no shortage of projects...). Everything's working in the terminal > (even a read-write $HOME, which we didn't have previously), and I'm > pretty happy with that! > > All suggestions were read and are appreciated, even though I'm only > quoting the one we ended up going with. Thank you Kirill, MRAB, Abdur, > and Johannes! You're welcome! Take a look at the "Kivy Launcher" App for Android if You don't want to do deep learning of Kivy. With this App You can just copy some Python code to the Kivy launcher directory "/sdcard/kivy/" and execute it with just one click without packaging your own APK file. Don't know which Python version is included in Kivy Launcher and believe it is 2.7. but it think Kivy will go over to Python 3.* in the near future. Johannes From steve+comp.lang.python at pearwood.info Tue Feb 20 18:15:44 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 23:15:44 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> Message-ID: On Tue, 20 Feb 2018 17:11:05 +0000, Wild, Marcel, Prof wrote: > I scarcely know Python, and I have no intention delving into it further. > I was forced to use Python because it features binary decision > diagrams, which MATHEMATICA doesn't. Coming from Mathematica the > account of Nathan Murphy reads like a nightmare. I dare say that was Nathan Murphy's intention. But the ironic thing is that by Murphy's standards, Mathematica suffers the same flaws as Python: it to is a dynamically typed language with little or no compile-time type safety. -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 20 18:17:13 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 23:17:13 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> Message-ID: On Tue, 20 Feb 2018 22:19:13 +0100, Christian Gollwitzer wrote: [...] > LeapYearCheck could be implemented using template metaprogramming > (quite horrible) or the new funky constexpr feature in C++11/C++14 (less > horrible). Thanks Christian. That's certainly interesting, I don't know much about template metaprogramming so TIL. -- Steve From steve+comp.lang.python at pearwood.info Tue Feb 20 18:25:26 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 23:25:26 +0000 (UTC) Subject: How to transform this as a service References: Message-ID: On Tue, 20 Feb 2018 14:41:42 -0800, Maroso Marco wrote: > My program seems to have a problem and i don't understand which! Firstly > it worked and now just went to run it again and it doesnt. [...] > Can someone please tell me how to transform it in a service to run on > Windows? There is absolutely no point in spending time and effort transforming it to a service if your program doesn't work. First you need to fix the program so that it can reliably and repeatedly work. Start by explaining, what do you mean by "doesn't work"? Does it: - raise an exception; - silently sit and do nothing for hours; - cause the computer to catch fire; - something else? Explain what you expect the script to do, and what it actually does instead. If there is an exception or error, please COPY AND PASTE the EXACT error message, if possible. Don't retype it from memory, summarise it, or take a screen shot. How are you running your script? To make it usable as a service, first you have to make sure it is usable from the system command prompt (command.com, cmd.exe, PowerShell, or whatever you use on Windows these days). Are you sure you are running your script using the right Python version? How do you know? -- Steve From mailman at hanez.org Tue Feb 20 18:33:10 2018 From: mailman at hanez.org (Johannes Findeisen) Date: Wed, 21 Feb 2018 00:33:10 +0100 Subject: Python on Android? In-Reply-To: <20180221001124.16be9365@orbit.fritz.box> References: <20180218175734.6e8908d9@orbit.fritz.box> <20180221001124.16be9365@orbit.fritz.box> Message-ID: <20180221003310.46ddf6db@orbit.fritz.box> On Wed, 21 Feb 2018 00:11:24 +0100 Johannes Findeisen wrote: > On Tue, 20 Feb 2018 23:12:23 +1100 > Chris Angelico wrote: > > > On Mon, Feb 19, 2018 at 3:57 AM, Johannes Findeisen wrote: > > > On Sun, 18 Feb 2018 20:57:02 +1100 > > > Chris Angelico wrote: > > > > > >> Does anyone have experience with running Python scripts on Android > > >> phones? I have a brother (honestly! I'm not actually using a phone > > >> myself!) who's trying to run one of my scripts in QPython, which > > >> claims to be version 3.2.2. I think that really truly is a Python 3.2 > > >> implementation - probing for newer features suggests that it actually > > >> doesn't even support the u"..." syntax that came (back) in with Python > > >> 3.3. So... does anyone know of a Python interpreter that's compatible > > >> with 3.4 or better and runs on Android? > > >> > > > > > > There is an App for Android called "Pydroid 3". You can find it in > > > the Google Play Store [0]. It provides a Python interpreter in version > > > 3.6.2 in its current release. > > > > > > The Python binary is installed under > > > > > > /data/user/0/ru.iiec.pydroid3/files/arm-linux-androideabi/bin/python > > > > > > but I can not access it in a normal terminal without being user "root" > > > but it is usable from within the terminal included in the Pydroid App. > > > > Many thanks for all the suggestions. Pydroid is the one that has > > worked; he's now able to run stuff happily from the terminal. Next > > step is to create an icon that invokes it, but that's looking rather > > harder (probably means I'm going to have to learn Kivy, and I already > > have no shortage of projects...). Everything's working in the terminal > > (even a read-write $HOME, which we didn't have previously), and I'm > > pretty happy with that! > > > > All suggestions were read and are appreciated, even though I'm only > > quoting the one we ended up going with. Thank you Kirill, MRAB, Abdur, > > and Johannes! > > You're welcome! > > Take a look at the "Kivy Launcher" App for Android if You don't want to > do deep learning of Kivy. With this App You can just copy some > Python code to the Kivy launcher directory "/sdcard/kivy/" > and execute it with just one click without packaging your own APK file. > > Don't know which Python version is included in Kivy Launcher and believe > it is 2.7. but it think Kivy will go over to Python 3.* in the near > future. I forgot the link to the documentation... https://kivy.org/docs/guide/packaging-android.html#packaging-your-application-for-kivy-launcher Johannes From steve+comp.lang.python at pearwood.info Tue Feb 20 18:43:00 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 20 Feb 2018 23:43:00 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Tue, 20 Feb 2018 12:42:23 -0800, Rick Johnson wrote: > For instance, if the age is queried many times a second, it would be a > much wiser design to set-up an event that will advance the age at the > end of the last second of every year Do you really mean to say that everybody in the world has their birthday on January 1st? We're not racehorses you know. Under your scheme, 99.7% of records will return the wrong age (off by one) at least once per year. Statistically, 50% of queries will be wrong. -- Steve From ben.usenet at bsb.me.uk Tue Feb 20 19:59:23 2018 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 21 Feb 2018 00:59:23 +0000 Subject: could use some help with this problem! (Posting On Python-List Prohibited) References: <8a1de4c4-915a-4147-89f1-faca7ef62b0c@googlegroups.com> <87371v4urk.fsf@bsb.me.uk> Message-ID: <87tvub17kk.fsf@bsb.me.uk> Lawrence D?Oliveiro writes: > On Wednesday, February 21, 2018 at 3:10:25 AM UTC+13, Ben Bacarisse wrote: >> You almost never /have/ to use nested loops. Has the course got this >> far without introducing the idea of a function? > > If you are using a function to avoid a nested loop, perhaps that?s not > such a smart use of a function... Agreed. And yet it /might/ be a smart use of one. Nothing about simply avoiding a loop is sufficient to make the assessment. -- Ben. From steve+comp.lang.python at pearwood.info Tue Feb 20 23:13:57 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 21 Feb 2018 04:13:57 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> <0ceaa666-a4c2-0562-d02d-8f85baa24ba9@vub.be> Message-ID: On Tue, 20 Feb 2018 10:17:12 -0700, Ian Kelly wrote: > On Tue, Feb 20, 2018 at 8:38 AM, Steven D'Aprano > wrote: >> On Tue, 20 Feb 2018 15:23:44 +0100, Antoon Pardon wrote: >> >>>> Okay. Now create a constraint on a name in C++ such that it can only >>>> accept integers representing A.D. years which, on the Gregorian >>>> calendar, are leap years. (Using a dedicated integer-like type is >>>> permitted.) It must accept all multiples of four, except those which >>>> are multiples of one hundred, unless they're also multiples of four >>>> hundred. >>>> >>>> That's what Steve asked for. Can you do it? Or is the C++ type system >>>> not flexible enough for that? >>> >>> Steve had multiple contributions in this thread. I didn't react to the >>> one where he asked for that. >> >> Yes you did: you refused to meet the challenge, stating (and I quote): >> >> "Why should this be done at compile time?" >> >> https://mail.python.org/pipermail/python-list/2018-February/730995.html > > I really don't understand what point you're driving at here, Steven. To be perfectly frank, neither do I any more. I fear I've been suckered into taking a position I didn't intend to, as often happens when I reply to Antoon Pardon. Obviously both statically and dynamically typed languages are Turing Complete, so any constraint you can apply at run-time in one you can apply at run-time in the other. How *easy* that is depends on the language features, and particularly for older languages, statically typed languages tend to be harder and less convenient to write in. There's typically more boilerplate, and more time spent placating the type- checker. Do I need to justify this or can we take it as a given? So I didn't think I was taking a controversial position to say that dynamic languages are good for writing constraints that are enforced at run-time, *as opposed to trying to do so within the type-system* which was the topic under discussion. I have argued in the past that the hard distinction between static and dynamic languages has been gradually worn away (if it ever existed at all!) as statically-typed languages add dynamic features, and dynamically- typed languages add static features. For example, Java supports run-time method dispatch; Python added type annotations to standardise on syntax for static type testing. Antoon's example of Pascal range checking is another run-time feature (a form of dynamic typing, in a fifty year old statically typed language no less!), as are Eiffel pre- and post-condition assertions (although the compiler can optimize them away if it can determine that they always hold). I've linked to Steve Yegge a lot, e.g.: https:// steve-yegge.blogspot.com.au/2008/05/dynamic-languages-strike-back.html so I certainly know that there's a certain amount of convergence between static and dynamic features and it was never my intention to suggest that statically-typed code can't validate values at run-time. That would be a ludicrous position to take. If Antoon was arguing in good faith surely he must have realised I couldn't have meant that. By all means call me out on factual inaccuracies, but do so in good faith. Don't assume I mean something ludicrous. If you look back at my reply to Bart: https://mail.python.org/pipermail/python-list/2018-February/730943.html you will see that the context is a discussion comparing: compile-time static analysis versus run-time dynamic checks strategies. Hence my challenge to perform the same kind of check at compile-time using an actual, existing type-system. (Not merely say a sufficiently clever type-system can do it and handwave away the practical difficulties.) Of course we can perform arbitrarily complex run-time checks in a language with static typing (it might not be as convenient or easy, especially if you have to declare every single temporary variable, do a lot of explicit casts, never re-use any variable, and use a lot of verbose boilerplate, but it can be done). > The > original claim (by you) was that dynamic languages excel at "enforcing > constraints at run-time which are hard to enforce at compile-time". The > counter-argument, as I understand, it was that while the constraints may > be hard to enforce at compile-time, they are just as easy to enforce at > run-time in a static language as in a dynamic language, so that can't > really be considered a *strength* per se of dynamic languages. Fair enough -- I acknowledge there's something to what you say, even if I wouldn't say it that way. But in which case, if dynamically typed languages are no easier to use than statically typed ones, and no faster, and no more secure, why do people use them? The conventional answer is that they are easier to use, hence more rapid prototyping and application development. But if we say that static typed languages are *just as easy to use* then the existence and popularity of Python, Ruby, Javascript, Perl, Smalltalk, Lisp, Scheme etc become a mystery. Why do we bother? > You then > followed this up by issuing a challenge to enforce this as a > compile-type check in C++. [Pedant: I never said C++, I said the choice of his language.] Looking back, I see that I misinterpreted Antoon's response. He is not disagreeing with me in the way I thought. I feel that Antoon has a long history of what *seems to me* to be petty point scoring, where he will make statements which seem carefully crafted to appear to be more extreme than they really are, so as to encourage misinterpretation and set off a response. He and I have a long history where I will make a statement which is not absolutely 100% pedantically correct, and Antoon will respond in a curt, peremptory fashion using language which can be read as taking a more extreme position than a more literal, careful reading will show. And I keep falling for it :-( To sketch a caricature of the process, *as I see it*: Me: "... and as we all know, the world is spherical, not flat." Antoon: "What are you talking about? The world isn't spherical." Me: "Of course it is! If not, how do you explain the observation that the hull of sailing ships disappear from sight before their sails?" Antoon: "I don't have to explain that. It has nothing to do with my argument." Me: "WTF???" [and finally, twenty posts later...] Me: "Wait a second... you didn't actually say the world was flat. Are you talking about the world being an oblate spheroid?" Antoon: "No." Me: *penny drops* "Do you mean the world is within 0.001% of an oblate spheroid?" Antoon: "Of course. What did you think I meant?" (If you search the archives, you'll find other examples of this, but what you won't see are the many, many, many draft posts I've written in response to Antoon which I've never sent because I decided there was simply no point arguing.) I'm a pedant myself, and I don't mind being corrected when I'm wrong if the correction is justified by the facts. But I cannot help interpreting Antoon's style of discourse as petty point-scoring, for the sake of disagreement rather than for the sake of education. A nanometre short of trolling. If I have misinterpreted Antoon's intentions, I apologise, but I can only be honest about the impression I am getting. Antoon, if you're reading this, and if you are *not* trying to point- score, then you ought to work on your communication skills. -- Steve From rosuav at gmail.com Wed Feb 21 00:09:51 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Feb 2018 16:09:51 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <9E15048A-2265-42F3-868B-98B138747F39@sun.ac.za> References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <9E15048A-2265-42F3-868B-98B138747F39@sun.ac.za> Message-ID: On Wed, Feb 21, 2018 at 6:39 AM, Geldenhuys, J, Prof wrote: > I think your case illustrates the Python/Mathematica issue well: you found a job for which Mathematica was not the perfect tool and you used Python. At the end of the day, both M & P have their place. For example, we probably won't use either to teach Introduction Computer Science soon, because they both lack features that we expect our students to be familiar with at the end of their first year. > Out of curiosity, what features does Python lack in that area? (I don't know anything about Mathematica, and it's only tangentially on-topic at best.) For an intro to comp sci, I would generally expect to start with a high level language such as Python, or JavaScript (because of its ubiquity, primarily due to web browser usage), or Ruby (particularly if you're Japanese). You don't learn about how to manage memory (because it's done for you), but on the other hand, you don't learn about how to manage memory (because hey, it's all done for you!). You don't learn how to wade through a crash dump (because you have exceptions and tracebacks), you don't learn how to compile for different platforms, you don't learn all sorts of other things that aren't necessary for someone's first year in comp sci. So I'm interested to what such high level languages lack, from the POV of a first year of comp sci. ChrisA From tjreedy at udel.edu Wed Feb 21 00:18:47 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Feb 2018 00:18:47 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 2/20/2018 8:38 AM, Antoon Pardon wrote: > People praise the dynamic nature of Python here on this list and then > often enough seem to recoil when they see a piece of code really using > that dynamism. Dynamic typing is the addition of run-time type information (RTTI) to data values. This allows duck typing of function parameters and function code. Every function that uses duck typing, which is to say, most functions written in Python, is 'really using that dynamism' as intended. The concrete type of arguments may change with every call, but the function code uses run-time type dispatch to get type-specific versions of the operations needed. For instance, builtin min has a single parameter whose abstract argument type is "iterable of '<'-compatible objects". It's code somewhere compares current_min < next_item, which dispatches to current_min.__lt__ or possibly next_item.__ge__. Combining "list of | tuple of | set of | frozenset of | iterable of | dict keyed with" witn "ints | doubles | strings | bytes | '<'-compatible lists | '<'-compatible tuples" gives 36 possible concrete input types. "array of x" gives an indefinite number more, as does user additions. I really like that we do not have to define a 'min_collection_type' function for every combination we might want to use in a package. When makes people recoil is abusing dynamism by needlessly rebinding a name to objects of different specific type within a single block of code. -- Terry Jan Reedy From tjreedy at udel.edu Wed Feb 21 00:25:29 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Feb 2018 00:25:29 -0500 Subject: could use some help with this problem! (Posting On Python-List Prohibited) In-Reply-To: <87tvub17kk.fsf@bsb.me.uk> References: <8a1de4c4-915a-4147-89f1-faca7ef62b0c@googlegroups.com> <87371v4urk.fsf@bsb.me.uk> <87tvub17kk.fsf@bsb.me.uk> Message-ID: On 2/20/2018 7:59 PM, Ben Bacarisse wrote: > Lawrence D?Oliveiro writes: > >> On Wednesday, February 21, 2018 at 3:10:25 AM UTC+13, Ben Bacarisse wrote: >>> You almost never /have/ to use nested loops. Has the course got this >>> far without introducing the idea of a function? >> >> If you are using a function to avoid a nested loop, perhaps that?s not >> such a smart use of a function... > > Agreed. And yet it /might/ be a smart use of one. Nothing about simply > avoiding a loop is sufficient to make the assessment. If one were using functional style, one would use tail recursive functions instead of *any* explicit loops. -- Terry Jan Reedy From arj.python at gmail.com Wed Feb 21 00:44:56 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Wed, 21 Feb 2018 09:44:56 +0400 Subject: Python on Android? In-Reply-To: References: Message-ID: here is a kivy launcher tutorial i once wrote : https://wp.me/p7UB6x-kB Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ On 18 Feb 2018 13:59, "Chris Angelico" wrote: Does anyone have experience with running Python scripts on Android phones? I have a brother (honestly! I'm not actually using a phone myself!) who's trying to run one of my scripts in QPython, which claims to be version 3.2.2. I think that really truly is a Python 3.2 implementation - probing for newer features suggests that it actually doesn't even support the u"..." syntax that came (back) in with Python 3.3. So... does anyone know of a Python interpreter that's compatible with 3.4 or better and runs on Android? Personal experiences trump just searching the web... ChrisA -- https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Wed Feb 21 00:51:04 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Feb 2018 16:51:04 +1100 Subject: Python on Android? In-Reply-To: References: Message-ID: On Wed, Feb 21, 2018 at 4:44 PM, Abdur-Rahmaan Janhangeer wrote: > here is a kivy launcher tutorial i once wrote : > https://wp.me/p7UB6x-kB > Thanks. I'm currently a dozen or so tabs deep into learning Kivy, and am just starting to get to looking into launchers. ChrisA From arj.python at gmail.com Wed Feb 21 00:56:35 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Wed, 21 Feb 2018 09:56:35 +0400 Subject: Python on Android? In-Reply-To: References: Message-ID: writing this especially to thank you hey XD pydroid3 suports qt5 and matplotlib o_0 numpy and sci-kit really, what an amazing discovery !!! Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ On 18 Feb 2018 13:59, "Chris Angelico" wrote: > Does anyone have experience with running Python scripts on Android > phones? I have a brother (honestly! I'm not actually using a phone > myself!) who's trying to run one of my scripts in QPython, which > claims to be version 3.2.2. I think that really truly is a Python 3.2 > implementation - probing for newer features suggests that it actually > doesn't even support the u"..." syntax that came (back) in with Python > 3.3. So... does anyone know of a Python interpreter that's compatible > with 3.4 or better and runs on Android? > > Personal experiences trump just searching the web... > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From antoon.pardon at vub.be Wed Feb 21 02:58:39 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Wed, 21 Feb 2018 08:58:39 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> <0ceaa666-a4c2-0562-d02d-8f85baa24ba9@vub.be> Message-ID: On 20-02-18 16:38, Steven D'Aprano wrote: > On Tue, 20 Feb 2018 15:23:44 +0100, Antoon Pardon wrote: > >>> Okay. Now create a constraint on a name in C++ such that it can only >>> accept integers representing A.D. years which, on the Gregorian >>> calendar, are leap years. (Using a dedicated integer-like type is >>> permitted.) It must accept all multiples of four, except those which >>> are multiples of one hundred, unless they're also multiples of four >>> hundred. >>> >>> That's what Steve asked for. Can you do it? Or is the C++ type system >>> not flexible enough for that? >> Steve had multiple contributions in this thread. I didn't react to the >> one where he asked for that. > Yes you did: you refused to meet the challenge, stating (and I quote): Look, you stated at a certain point that dynamic languages really excelled at run time checks. That suggests that static languages don't. So when I point out that static languages are just as good at run time checks, you refer about how you want static languages to do their checks at compile time. But the fact that you want static languages to do those things a compile time, is not a good reason to claim that dynamic languages really excell at run time checks, when in fact static languages are just as good at doing run time checks. -- Antoon Pardon. From antoon.pardon at vub.be Wed Feb 21 03:09:36 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Wed, 21 Feb 2018 09:09:36 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> <2216a43c-0765-acfa-a831-20454d84c25a@vub.be> <0ceaa666-a4c2-0562-d02d-8f85baa24ba9@vub.be> Message-ID: <711ef3f2-def4-2dd9-2ab3-d7290ff8fd8f@vub.be> On 21-02-18 05:13, Steven D'Aprano wrote: > On Tue, 20 Feb 2018 10:17:12 -0700, Ian Kelly wrote: > >> On Tue, Feb 20, 2018 at 8:38 AM, Steven D'Aprano >> wrote: >>> On Tue, 20 Feb 2018 15:23:44 +0100, Antoon Pardon wrote: >>> >>>>> Okay. Now create a constraint on a name in C++ such that it can only >>>>> accept integers representing A.D. years which, on the Gregorian >>>>> calendar, are leap years. (Using a dedicated integer-like type is >>>>> permitted.) It must accept all multiples of four, except those which >>>>> are multiples of one hundred, unless they're also multiples of four >>>>> hundred. >>>>> >>>>> That's what Steve asked for. Can you do it? Or is the C++ type system >>>>> not flexible enough for that? >>>> Steve had multiple contributions in this thread. I didn't react to the >>>> one where he asked for that. >>> Yes you did: you refused to meet the challenge, stating (and I quote): >>> >>> "Why should this be done at compile time?" >>> >>> https://mail.python.org/pipermail/python-list/2018-February/730995.html >> I really don't understand what point you're driving at here, Steven. > To be perfectly frank, neither do I any more. I fear I've been suckered > into taking a position I didn't intend to, as often happens when I reply > to Antoon Pardon. > > Obviously both statically and dynamically typed languages are Turing > Complete, so any constraint you can apply at run-time in one you can > apply at run-time in the other. How *easy* that is depends on the > language features, and particularly for older languages, statically typed > languages tend to be harder and less convenient to write in. There's > typically more boilerplate, and more time spent placating the type- > checker. Do I need to justify this or can we take it as a given? > > So I didn't think I was taking a controversial position to say that > dynamic languages are good for writing constraints that are enforced at > run-time, *as opposed to trying to do so within the type-system* which > was the topic under discussion. Yes it is controversial. You seem to conflate "within the type-system" with "at compile time". If you want to make a general statement of dynamically typed languages vs statically typed languages, I don't think you can do that. There are plenty of statically typed languages that also do run time checks. I didn't sucker you into a position. Your inaccurate wording suckered you into a position. -- Antoon. From antoon.pardon at vub.be Wed Feb 21 03:15:50 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Wed, 21 Feb 2018 09:15:50 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 21-02-18 06:18, Terry Reedy wrote: > On 2/20/2018 8:38 AM, Antoon Pardon wrote: > >> People praise the dynamic nature of Python here on this list and then >> often enough seem to recoil when they see a piece of code really using >> that dynamism. > > ... > > When makes people recoil is abusing dynamism by needlessly rebinding a > name to objects of different specific type within a single block of code. > But that was what Steven was complaining about. The fact that in a static language He would need two names if he wanted a variable in a single block of code to be first a number and then a string. At least that is how I understood him. -- Antoon. From tkadm30 at yandex.com Wed Feb 21 04:13:56 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 21 Feb 2018 04:13:56 -0500 Subject: Python to Julia code generator? Message-ID: <37752134-4368-b74c-7744-6cf07c612e37@yandex.com> Hi, Would it be possible to build a Python to Julia code generator?? i'm interested to learn Julia and would love to have the capacity to embed or run native Python code in Julia.. Thx Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From steve+comp.lang.python at pearwood.info Wed Feb 21 05:04:44 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 21 Feb 2018 10:04:44 +0000 (UTC) Subject: Python to Julia code generator? References: <37752134-4368-b74c-7744-6cf07c612e37@yandex.com> Message-ID: On Wed, 21 Feb 2018 04:13:56 -0500, Etienne Robillard wrote: > Hi, > > Would it be possible to build a Python to Julia code generator?? > > i'm interested to learn Julia and would love to have the capacity to > embed or run native Python code in Julia.. http://blog.leahhanson.us/post/julia/julia-calling-python.html -- Steve From tkadm30 at yandex.com Wed Feb 21 05:25:09 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 21 Feb 2018 05:25:09 -0500 Subject: Python to Julia code generator? In-Reply-To: References: <37752134-4368-b74c-7744-6cf07c612e37@yandex.com> Message-ID: <8defb6b1-fcc4-9380-6bad-316a7d42e279@yandex.com> I found this: https://github.com/JuliaPy/PyCall.jl Looks pretty awesome already! :-) Thx E Le 2018-02-21 ? 05:04, Steven D'Aprano a ?crit?: > On Wed, 21 Feb 2018 04:13:56 -0500, Etienne Robillard wrote: > >> Hi, >> >> Would it be possible to build a Python to Julia code generator?? >> >> i'm interested to learn Julia and would love to have the capacity to >> embed or run native Python code in Julia.. > http://blog.leahhanson.us/post/julia/julia-calling-python.html > > > > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From rosuav at gmail.com Wed Feb 21 05:27:41 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 21 Feb 2018 21:27:41 +1100 Subject: Python to Julia code generator? In-Reply-To: References: <37752134-4368-b74c-7744-6cf07c612e37@yandex.com> Message-ID: On Wed, Feb 21, 2018 at 9:04 PM, Steven D'Aprano wrote: > On Wed, 21 Feb 2018 04:13:56 -0500, Etienne Robillard wrote: > >> Hi, >> >> Would it be possible to build a Python to Julia code generator?? >> >> i'm interested to learn Julia and would love to have the capacity to >> embed or run native Python code in Julia.. > > http://blog.leahhanson.us/post/julia/julia-calling-python.html > """If you find that your production code is too slow because you?re using mutual recursion between nine different languages, blame Dan Luu for this terrible idea.""" I have... NEVER gone as far as nine. That takes the cake. In fact, I don't recall ever going beyond three. At least, not in production... ChrisA From tjreedy at udel.edu Wed Feb 21 05:31:15 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Feb 2018 05:31:15 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 2/21/2018 3:15 AM, Antoon Pardon wrote: > On 21-02-18 06:18, Terry Reedy wrote: >> On 2/20/2018 8:38 AM, Antoon Pardon wrote: >> >>> People praise the dynamic nature of Python here on this list and then >>> often enough seem to recoil when they see a piece of code really using >>> that dynamism. >> >> ... >> >> When makes people recoil is abusing dynamism by needlessly rebinding a >> name to objects of different specific type within a single block of code. >> > But that was what Steven was complaining about. The fact that in a static language > He would need two names if he wanted a variable in a single block of code to be > first a number and then a string. At least that is how I understood him. I am not a party to any dispute between you and Steven. I intentionally inserted 'needlessly' in that comment to cover the situation where a competent Python programmer has at least a plausible reason for rebinding within a block. But in Python, this is rare compared to the constant binding of parameters names to whatever argument one passes. Types indicate how to treat a particular object or, in some computer languages, a block of memory. Declaring a type for a name that is permanently attached to an information entity is an indirect way of associating a type with the entity. If entities are tagged with their type, then names need not be. -- Terry Jan Reedy From tkadm30 at yandex.com Wed Feb 21 05:35:27 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 21 Feb 2018 05:35:27 -0500 Subject: Python to Julia code generator? In-Reply-To: References: <37752134-4368-b74c-7744-6cf07c612e37@yandex.com> Message-ID: <380cc532-cde0-6d50-1d99-395b50bbb1ff@yandex.com> Le 2018-02-21 ? 05:27, Chris Angelico a ?crit?: > > """If you find that your production code is too slow because you?re > using mutual recursion between nine different languages, blame Dan Luu > for this terrible idea.""" > > I have... NEVER gone as far as nine. That takes the cake. In fact, I > don't recall ever going beyond three. At least, not in production... > > ChrisA Dude that is advanced stuff... :-) Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From lorenzo.gatti at gmail.com Wed Feb 21 05:50:26 2018 From: lorenzo.gatti at gmail.com (lorenzo.gatti at gmail.com) Date: Wed, 21 Feb 2018 02:50:26 -0800 (PST) Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <87mv07zwhj.fsf@bsb.me.uk> References: <877erbsxak.fsf@elektro.pacujo.net> <87mv07zwhj.fsf@bsb.me.uk> Message-ID: On Saturday, February 17, 2018 at 12:28:29 PM UTC+1, Ben Bacarisse wrote: > Marko Rauhamaa writes: > > > Many people think static typing is key to high quality. I tend to think > > the reverse is true: the boilerplate of static typing hampers > > expressivity so much that, on the net, quality suffers. > > I don't find that with Haskell. It's statically typed but the types are > almost always inferred. If you see an explicit type, it's usually > because the author thinks it helps explain something. > > (I don't want to start a Haskell/Python thread -- the only point is that > static typing does not inevitably imply lots of 'boilerplate'.) > > -- > Ben. There are two sides to not declaring types: having readers spend a fraction of a second to figure out what types are being used and having tools apply type inference for useful purposes. Python is bad at type inference (but only because deliberate loopholes like eval() are preserved) but good at making programmers trust code, while Haskell is bad at encouraging straightforward and understandable types but good at extracting maximum value from type inference. From antoon.pardon at vub.be Wed Feb 21 06:18:58 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Wed, 21 Feb 2018 12:18:58 +0100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <871599f4-4879-cb2e-bad4-9c9885885ae8@vub.be> Message-ID: On 21-02-18 11:31, Terry Reedy wrote: > On 2/21/2018 3:15 AM, Antoon Pardon wrote: >> On 21-02-18 06:18, Terry Reedy wrote: >>> On 2/20/2018 8:38 AM, Antoon Pardon wrote: >>> >>>> People praise the dynamic nature of Python here on this list and then >>>> often enough seem to recoil when they see a piece of code really using >>>> that dynamism. >>> >>> ... >>> >>> When makes people recoil is abusing dynamism by needlessly rebinding a >>> name to objects of different specific type within a single block of >>> code. >>> >> But that was what Steven was complaining about. The fact that in a >> static language >> He would need two names if he wanted a variable in a single block of >> code to be >> first a number and then a string. At least that is how I understood him. > > I am not a party to any dispute between you and Steven. But that was still the context in which the above contributions were made. > > I intentionally inserted 'needlessly' in that comment to cover the > situation where a competent Python programmer has at least a plausible > reason for rebinding within a block.? But in Python, this is rare > compared to the constant binding of parameters names to whatever > argument one passes. I find 'needlessly' is without real meaning here. What one person finds needless an other finds interesting. What one finds annoying insertions to placate the type system, an other finds an interesting way to make intentions clear and an implicite way to let the runtime check boundareis when needed. The designer of Python have made their choice, fine I can live with their choice and a lot of people even seem happy with that choice. That is fine too. It doesn't mean we need to react as if all can be said about the alternatives is how annoying they are. > > Types indicate how to treat a particular object or, in some computer > languages, a block of memory.? Declaring a type for a name that is > permanently attached to an information entity is an indirect way of > associating a type with the entity.? If entities are tagged with their > type, then names need not be. > That they don't need to be, doesn't contradict it could be usefull. If I have the intention that a certain name will only ever be bound to numeric values between -10 and +10. Then IMO it would be usefull if one could express this in the language and let the language runtime do the needed checks each time this name is rebound. It certainly would be less annoying than repeatedly having to insert the same extra code that checks for these restraints. -- Antoon. From Richard at Damon-Family.org Wed Feb 21 08:13:57 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Wed, 21 Feb 2018 08:13:57 -0500 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On 2/20/18 3:51 PM, Chris Angelico wrote: > On Wed, Feb 21, 2018 at 7:42 AM, Rick Johnson > wrote: >> On Tuesday, February 20, 2018 at 2:18:31 PM UTC-6, MRAB wrote: >> >>> The point he was making is that if you store a person's age, you'd have >>> to update it every year. It's far better to store the date of birth and >>> calculate the age on demand. >> *AHEM* >> >> At the risk of being labeled a "quibbler" (which, in the >> grander scheme is not really all that bad considering some >> of the names that have been attributed to me), i must say >> your advice is only "sound advice" in cases where the age >> will not be queried frequently. >> >> For instance, if the age is queried many times a second, it >> would be a much wiser design to set-up an event that will >> advance the age at the end of the last second of every year, >> instead of doing the age calculation many times a second. >> Heck, even if the frequency is multiple time a day, a valid >> argument could be made. > Nope. Even if you need the age many times per second, it's still > better to store the date of birth, because you eliminate boundary > conditions and duplicated data. But that's assuming you're storing the > age of *a person*. If the age in question is a boundary ("motor > insurance premiums are increased if the driver is 65 years or older"), > then it's still sane to store the age. > > ChrisA I would normally expect that the persistent store would have a birth date of the person, but within a local operation, you likely want to be able to assume the consistency of a attribute like over 55 years old, so you would compute the age, for that operation, but not keep it beyond that. Otherwise, depending on the order of doing your checks you may find that during your processing they are either both or neither of under 55 and over 55. It is much easier to reason that your code is correct when you can prevent (and thus ignore) such data races. This normally means your operation grabs the value of 'now' once, and all processing will be based on THAT point of time, not what currently is 'now'. How big this 'timestop' needs to be depends on how wide the requirement is for temporal consistency. It might be just within a function, it might be for the full duration of the program run, 'now' might even be a parameter to the program, i.e. as of last (or next) midnight, do the computation. -- Richard Damon From none at gmail.com Wed Feb 21 08:27:54 2018 From: none at gmail.com (ast) Date: Wed, 21 Feb 2018 14:27:54 +0100 Subject: Writing some floats in a file in an efficient way Message-ID: <5a8d73e0$0$7175$426a34cc@news.free.fr> Hello I would like to write a huge file of double precision floats, 8 bytes each, using IEEE754 standard. Since the file is big, it has to be done in an efficient way. I tried pickle module but unfortunately it writes 12 bytes per float instead of just 8. Example: import pickle f = open("data.bin", "wb") mypickler = pickle.Pickler(f) mypickler.dump(0.0) mypickler.dump(-0.0) f.close() Let's see what file data.bin contains now: 80 03 47 00 00 00 00 00 00 00 00 2E 80 03 47 80 00 00 00 00 00 00 00 2E We see our 2 floats 00 00 00 00 00 00 00 00 which is the IEEE754 representation for 0.0 and 80 00 00 00 00 00 00 00 which is the IEEE754 representation for -0.0 yes, there are two 0 for floats, a positive and a negative one ;-) but there is a 3 bytes overhead 80 03 47 and an ending byte 2E for each float. This is a 50% overhead. Is there a way to write a float with only 8 bytes ? Thx From bc at freeuk.com Wed Feb 21 09:02:38 2018 From: bc at freeuk.com (bartc) Date: Wed, 21 Feb 2018 14:02:38 +0000 Subject: Writing some floats in a file in an efficient way In-Reply-To: <5a8d73e0$0$7175$426a34cc@news.free.fr> References: <5a8d73e0$0$7175$426a34cc@news.free.fr> Message-ID: On 21/02/2018 13:27, ast wrote: > Hello > > I would like to write a huge file of double precision > floats, 8 bytes each, using IEEE754 standard. Since > the file is big, it has to be done in an efficient > way. Time efficient or space efficient? If the latter, how many floats are we talking about? > I tried pickle module but unfortunately it writes > 12 bytes per float instead of just 8. > > Example: > > import pickle > > f = open("data.bin", "wb") > mypickler = pickle.Pickler(f) > > mypickler.dump(0.0) > mypickler.dump(-0.0) I don't know about pickle, but before looking at other approaches, have you tried outputting the numbers more than one at a time? I found that list or tuple of 10 floats produced a file of 98 or 97 bytes, a somewhat smaller overhead. 10,000, about 90KB, a 10% overhead. -- bartc From __peter__ at web.de Wed Feb 21 09:33:11 2018 From: __peter__ at web.de (Peter Otten) Date: Wed, 21 Feb 2018 15:33:11 +0100 Subject: Writing some floats in a file in an efficient way References: <5a8d73e0$0$7175$426a34cc@news.free.fr> Message-ID: ast wrote: > Is there a way to write a float with only 8 bytes ? If you want to write the values one-by-one convert them to bytes with struct.pack() and then write the result. To write many values at once use array.array.tofile() or numpy.array.tofile(). From peter.heitzer at rz.uni-regensburg.de Wed Feb 21 09:51:14 2018 From: peter.heitzer at rz.uni-regensburg.de (Peter Heitzer) Date: 21 Feb 2018 14:51:14 GMT Subject: Writing some floats in a file in an efficient way References: <5a8d73e0$0$7175$426a34cc@news.free.fr> Message-ID: ast wrote: >Hello >I would like to write a huge file of double precision >floats, 8 bytes each, using IEEE754 standard. Since >the file is big, it has to be done in an efficient >way. >I tried pickle module but unfortunately it writes >12 bytes per float instead of just 8. >Is there a way to write a float with only 8 bytes ? Use struct.pack() -- Dipl.-Inform(FH) Peter Heitzer, peter.heitzer at rz.uni-regensburg.de From none at gmail.com Wed Feb 21 10:54:13 2018 From: none at gmail.com (ast) Date: Wed, 21 Feb 2018 16:54:13 +0100 Subject: Writing some floats in a file in an efficient way In-Reply-To: References: <5a8d73e0$0$7175$426a34cc@news.free.fr> Message-ID: <5a8d9627$0$3693$426a34cc@news.free.fr> Le 21/02/2018 ? 15:02, bartc a ?crit?: > On 21/02/2018 13:27, ast wrote: > > Time efficient or space efficient? space efficient > If the latter, how many floats are we talking about? 10^9 From none at gmail.com Wed Feb 21 10:59:30 2018 From: none at gmail.com (ast) Date: Wed, 21 Feb 2018 16:59:30 +0100 Subject: Writing some floats in a file in an efficient way In-Reply-To: <5a8d73e0$0$7175$426a34cc@news.free.fr> References: <5a8d73e0$0$7175$426a34cc@news.free.fr> Message-ID: <5a8d9764$0$3313$426a34cc@news.free.fr> Le 21/02/2018 ? 14:27, ast a ?crit?: struct.pack() as advised works fine. Exemple: >>> import struct >>> struct.pack(">d", -0.0) b'\x80\x00\x00\x00\x00\x00\x00\x00' before I read your answers I found a way with pickle >>> import pickle >>> pickle.dumps(-0.0)[3:-1] b'\x80\x00\x00\x00\x00\x00\x00\x00' but struct.pack() is better Thx From michael.stemper at gmail.com Wed Feb 21 11:34:08 2018 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Wed, 21 Feb 2018 10:34:08 -0600 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <87o9kl8to9.fsf@nightsong.com> References: <4qqiC.223282$kD1.96803@fx34.am4> <87o9kl8to9.fsf@nightsong.com> Message-ID: On 2018-02-18 22:55, Paul Rubin wrote: > Steven D'Aprano writes: >>>> "positive odd integers greater than 10 but less than 15003 divisible by >>>> 17 except for 850, 867 and 1394; or primes that aren't Mersenne >>>> primes".... >> It *could* be a type, if your type system was sufficiently flexible to >> allow you to specify something in that level of detail. Of course no >> existing type system is. > > Of course dependent types could do that (they have a type for every > proposition in constructive predicate calculus). You might also be able > to do it with Liquid Haskell's refinement types, though automatically > checking them might not be so easy. > >> That's an easy one: even Pascal in the 1970s could deal with enumerated >> types like the values 1, 3, 5, 7, 9. (I think.) > > Idunno about Pascal, but Ada has integer range types. Back in the early 1980s, I took a few courses involving Pascal, and it certainly supported subranges then. Quoting from my text[1]: A _scalar subrange_ data type is a data type composed of a specified range of any of the other standard or user- defined scalar types, except type REAL. We define a subrange type with a TYPE declaration of the following format. TYPE type-name = lowerlimit..upperlimit; [...] Examples of subrange declarations are: TYPE EXAMSCORES = 0..100; Of course, Pascal being Pascal, a function to return the sum of an array of INTEGER would refuse to return the sum of an array of EXAMSCORES.[2] [1] _An Introduction to Programming and Problem Solving With Pascal_; Schneider, Weingart, and Perlman; (C) 1978 [2] Kernighan examines a similar issue in Section 2.1 of "Why Pascal is Not My Favorite Programming Language", -- Michael F. Stemper You can lead a horse to water, but you can't make him talk like Mr. Ed by rubbing peanut butter on his gums. From michael.stemper at gmail.com Wed Feb 21 11:35:20 2018 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Wed, 21 Feb 2018 10:35:20 -0600 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: <87o9kl8to9.fsf@nightsong.com> References: <4qqiC.223282$kD1.96803@fx34.am4> <87o9kl8to9.fsf@nightsong.com> Message-ID: On 2018-02-18 22:55, Paul Rubin wrote: > Steven D'Aprano writes: >>>> "positive odd integers greater than 10 but less than 15003 divisible by >>>> 17 except for 850, 867 and 1394; or primes that aren't Mersenne >>>> primes".... >> It *could* be a type, if your type system was sufficiently flexible to >> allow you to specify something in that level of detail. Of course no >> existing type system is. > > Of course dependent types could do that (they have a type for every > proposition in constructive predicate calculus). You might also be able > to do it with Liquid Haskell's refinement types, though automatically > checking them might not be so easy. > >> That's an easy one: even Pascal in the 1970s could deal with enumerated >> types like the values 1, 3, 5, 7, 9. (I think.) > > Idunno about Pascal, but Ada has integer range types. Back in the early 1980s, I took a few courses involving Pascal, and it certainly supported subranges then. Quoting from my text[1]: A _scalar subrange_ data type is a data type composed of a specified range of any of the other standard or user- defined scalar types, except type REAL. We define a subrange type with a TYPE declaration of the following format. TYPE type-name = lowerlimit..upperlimit; [...] Examples of subrange declarations are: TYPE EXAMSCORES = 0..100; Of course, Pascal being Pascal, a function to return the sum of an array of INTEGER would refuse to return the sum of an array of EXAMSCORES.[2] [1] _An Introduction to Programming and Problem Solving With Pascal_; Schneider, Weingart, and Perlman; (C) 1978 [2] Kernighan examines a similar issue in Section 2.1 of "Why Pascal is Not My Favorite Programming Language", References: <5a8d73e0$0$7175$426a34cc@news.free.fr> <5a8d9627$0$3693$426a34cc@news.free.fr> Message-ID: On 21/02/2018 15:54, ast wrote: > Le 21/02/2018 ? 15:02, bartc a ?crit?: >> On 21/02/2018 13:27, ast wrote: > >> >> Time efficient or space efficient? > > space efficient > >> If the latter, how many floats are we talking about? > > 10^9 OK. My experiment of writing the same 64-bit float a billion times to a file took 3 minutes, 8 bytes at a time. Doing it in blocks of 64KB, it still took 100 seconds (not using Python, and on Windows to a spinning hard drive, which is supposed to have slow file operations). So perhaps both time and space requirements need looking at. Was it just the extra overhead on top of the 8 billion bytes needed for those floats that you were worried about? If not, then you might look at some mild compression, but only if the data lends itself to it. For example, if there are lots of zeros, lots of integer values among the floats, or lots of consecutive repeated values. (And if precision is not so critical, you might also try just truncating a few bytes at the lower end. A bit crude, but it could be effective! You replace them with zeros when reading back in. However watch out for special floating point values, those involving underflow etc. Although it might be better to convert to proper 32-bit float format in this case. This will halve space and probably time requirements.) -- bartc From marcelo.huerta at gmail.com Wed Feb 21 12:30:51 2018 From: marcelo.huerta at gmail.com (MGHSM) Date: Wed, 21 Feb 2018 09:30:51 -0800 (PST) Subject: py2exe maintainer abandoned the project; is there any replacements? Or, anyone willing to continue the work? Message-ID: <7f0036e1-70b0-4759-8bc6-a7fac749015d@googlegroups.com> Thomas Heller himself says he's "retiring" from py2exe in https://sourceforge.net/p/py2exe/mailman/message/36033869/ Is there any suitable replacement with similar or better capabilities? Is there any desires to continue the project? From auriocus at gmx.de Wed Feb 21 12:38:19 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 21 Feb 2018 18:38:19 +0100 Subject: py2exe maintainer abandoned the project; is there any replacements? Or, anyone willing to continue the work? In-Reply-To: <7f0036e1-70b0-4759-8bc6-a7fac749015d@googlegroups.com> References: <7f0036e1-70b0-4759-8bc6-a7fac749015d@googlegroups.com> Message-ID: Am 21.02.18 um 18:30 schrieb MGHSM: > Thomas Heller himself says he's "retiring" from py2exe in https://sourceforge.net/p/py2exe/mailman/message/36033869/ > > Is there any suitable replacement with similar or better capabilities? > There is PyInstaller, which works on all major OS: http://www.pyinstaller.org/ Christian From kryptxy at protonmail.com Wed Feb 21 12:45:01 2018 From: kryptxy at protonmail.com (Kryptxy) Date: Wed, 21 Feb 2018 12:45:01 -0500 Subject: py2exe maintainer abandoned the project; is there any replacements? Or, anyone willing to continue the work? In-Reply-To: <7f0036e1-70b0-4759-8bc6-a7fac749015d@googlegroups.com> References: <7f0036e1-70b0-4759-8bc6-a7fac749015d@googlegroups.com> Message-ID: I like pyinstaller. It's one-file exe creation is pretty good. You can give it a try. -------- Original Message -------- On 21 Feb 2018, 23:00, MGHSM wrote: > Am 21.02.18 um 18:30 schrieb MGHSM: > Thomas Heller himself says he's "retiring" from py2exe in https://sourceforge.net/p/py2exe/mailman/message/36033869/ > > Is there any suitable replacement with similar or better capabilities? > There is PyInstaller, which works on all major OS: http://www.pyinstaller.org/ Christian -- https://mail.python.org/mailman/listinfo/python-list From ian.g.kelly at gmail.com Wed Feb 21 13:01:41 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 21 Feb 2018 11:01:41 -0700 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <9E15048A-2265-42F3-868B-98B138747F39@sun.ac.za> Message-ID: On Tue, Feb 20, 2018 at 10:09 PM, Chris Angelico wrote: > On Wed, Feb 21, 2018 at 6:39 AM, Geldenhuys, J, Prof > wrote: >> I think your case illustrates the Python/Mathematica issue well: you found a job for which Mathematica was not the perfect tool and you used Python. At the end of the day, both M & P have their place. For example, we probably won't use either to teach Introduction Computer Science soon, because they both lack features that we expect our students to be familiar with at the end of their first year. >> > > Out of curiosity, what features does Python lack in that area? (I > don't know anything about Mathematica, and it's only tangentially > on-topic at best.) For an intro to comp sci, I would generally expect > to start with a high level language such as Python, or JavaScript > (because of its ubiquity, primarily due to web browser usage), or Ruby > (particularly if you're Japanese). You don't learn about how to manage > memory (because it's done for you), but on the other hand, you don't > learn about how to manage memory (because hey, it's all done for > you!). You don't learn how to wade through a crash dump (because you > have exceptions and tracebacks), you don't learn how to compile for > different platforms, you don't learn all sorts of other things that > aren't necessary for someone's first year in comp sci. So I'm > interested to what such high level languages lack, from the POV of a > first year of comp sci. Indeed, MIT famously switched from Scheme to Python for their introductory class about a decade back, and as far as I'm aware they're still using it. From rosuav at gmail.com Wed Feb 21 16:14:18 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 22 Feb 2018 08:14:18 +1100 Subject: Python on Android? In-Reply-To: <20180221001124.16be9365@orbit.fritz.box> References: <20180218175734.6e8908d9@orbit.fritz.box> <20180221001124.16be9365@orbit.fritz.box> Message-ID: On Wed, Feb 21, 2018 at 10:11 AM, Johannes Findeisen wrote: > Don't know which Python version is included in Kivy Launcher and believe > it is 2.7. but it think Kivy will go over to Python 3.* in the near > future. > Well... after an insane number of attempts, most of which were at least partially successful, I'm finally pulling the plug on this project (for now). To get Python 3 support, I have to use unreleased versions of everything, which in turn means a heap of fiddling around in ways that aren't well documented ("how do you get X to use Y built from source?"). I think I'll leave this for six months or a year and see if the stable releases get support for Py3, at which point it'll be a lot easier to get everything going. But hey. I think there's enough here to suggest that this probably WOULD work. At least for a simple "hello, world" app; might not be enough for everything I'd like to do, but core functionality should be there. If anyone has any suggestions, here's the hello world project that I've been playing with: https://github.com/Rosuav/kivy-testing In theory, the command "buildozer android debug" ought to build an APK, but I didn't manage to get that to happen. If it works for you, can you please tell me ALL the packages you're using and their exact versions? I suspect that it's mainly a version problem. ChrisA From breamoreboy at gmail.com Wed Feb 21 23:18:39 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Thu, 22 Feb 2018 04:18:39 +0000 Subject: Computer History Museum Announces 2018 Fellow Award Honorees Message-ID: One is the BDFL http://www.computerhistory.org/press/2018-fellow-honorees.html -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From larry.martell at gmail.com Thu Feb 22 01:27:41 2018 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 22 Feb 2018 01:27:41 -0500 Subject: atws Message-ID: I want to use the atws package (https://atws.readthedocs.io/readme.html). I am using python 2.7.6 on ubuntu-trusty-64 3.13.0-87-generic. I get this error when importing the package: >>> import atws Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/atws/__init__.py", line 4, in from .wrapper import connect File "/usr/local/lib/python2.7/dist-packages/atws/wrapper.py", line 32, in from . import connection File "/usr/local/lib/python2.7/dist-packages/atws/connection.py", line 19, in from requests.exceptions import ConnectTimeout, Timeout, ReadTimeout, SSLError ImportError: cannot import name ConnectTimeout I would not be surprised if no one here has used this package, but has anyone seen this error in other packages or their own work? From breamoreboy at gmail.com Thu Feb 22 01:46:59 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Thu, 22 Feb 2018 06:46:59 +0000 Subject: atws In-Reply-To: References: Message-ID: On 22/02/18 06:27, Larry Martell wrote: > I want to use the atws package > (https://atws.readthedocs.io/readme.html). I am using python 2.7.6 on > ubuntu-trusty-64 3.13.0-87-generic. I get this error when importing > the package: > >>>> import atws > Traceback (most recent call last): > File "", line 1, in > File "/usr/local/lib/python2.7/dist-packages/atws/__init__.py", line > 4, in > from .wrapper import connect > File "/usr/local/lib/python2.7/dist-packages/atws/wrapper.py", line > 32, in > from . import connection > File "/usr/local/lib/python2.7/dist-packages/atws/connection.py", > line 19, in > from requests.exceptions import ConnectTimeout, Timeout, > ReadTimeout, SSLError > ImportError: cannot import name ConnectTimeout > > I would not be surprised if no one here has used this package, but has > anyone seen this error in other packages or their own work? > According to https://github.com/byt3bl33d3r/CrackMapExec/issues/167, which reports the same error, you just need to update the requests package. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Thu Feb 22 02:00:21 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 22 Feb 2018 18:00:21 +1100 Subject: atws In-Reply-To: References: Message-ID: On Thu, Feb 22, 2018 at 5:27 PM, Larry Martell wrote: > I want to use the atws package > (https://atws.readthedocs.io/readme.html). I am using python 2.7.6 on > ubuntu-trusty-64 3.13.0-87-generic. I get this error when importing > the package: > >>>> import atws > Traceback (most recent call last): > File "", line 1, in > File "/usr/local/lib/python2.7/dist-packages/atws/__init__.py", line > 4, in > from .wrapper import connect > File "/usr/local/lib/python2.7/dist-packages/atws/wrapper.py", line > 32, in > from . import connection > File "/usr/local/lib/python2.7/dist-packages/atws/connection.py", > line 19, in > from requests.exceptions import ConnectTimeout, Timeout, > ReadTimeout, SSLError > ImportError: cannot import name ConnectTimeout > > I would not be surprised if no one here has used this package, but has > anyone seen this error in other packages or their own work? You're running that on a fairly old version of Python (2.7.6 - the latest in the 2.x branch is 2.7.14). It's entirely possible that the app doesn't support the versions of Python and/or the 'requests' library that you have installed. First thing to try: can you use the app under Python 3? If you get the same error there, you may need to upgrade other packages as well. Installing 'requests' using pip in a virtual environment should get you a newer version and might solve your problem. For reference, here's the version of requests that I have (which does have that exception available): >>> import requests >>> requests.__version__ '2.18.4' What's yours? ChrisA From none at gmail.com Thu Feb 22 04:28:02 2018 From: none at gmail.com (ast) Date: Thu, 22 Feb 2018 10:28:02 +0100 Subject: Writing some floats in a file in an efficient way In-Reply-To: References: <5a8d73e0$0$7175$426a34cc@news.free.fr> <5a8d9627$0$3693$426a34cc@news.free.fr> Message-ID: <5a8e8d24$0$31633$426a74cc@news.free.fr> Le 21/02/2018 ? 18:23, bartc a ?crit?: > On 21/02/2018 15:54, ast wrote: >> Le 21/02/2018 ? 15:02, bartc a ?crit?: >>> On 21/02/2018 13:27, ast wrote: >> >>> >>> Time efficient or space efficient? >> >> space efficient >> >>> If the latter, how many floats are we talking about? >> >> 10^9 > > > Although it might be better to convert to proper 32-bit float format in > this case. This will halve space and probably time requirements.) > Yes, storing 32 bits only floats is a good idea. and the good news is that struct.pack() does the job. from struct import * >>> pack('>ff', 3.1234, 5.3e-7) b'@G\xe5\xc95\x0eES' # 2*4 bytes >>> unpack('>ff', b'@G\xe5\xc95\x0eES') (3.1233999729156494, 5.300000225361146e-07) What happens if we input some out of range floats ? >>> pack('>dd', 3.1e-500, 5.3e400) b'\x00\x00\x00\x00\x00\x00\x00\x00\x7f\xf0\x00\x00\x00\x00\x00\x00' >>> unpack('>dd', b'\x00\x00\x00\x00\x00\x00\x00\x00\x7f\xf0\x00\x00\x00\x00\x00\x00') (0.0, inf) So underflow/overflow are well handled I saw there is a half precision float too, on 16 bits, but it is really unaccurate From steve+comp.lang.python at pearwood.info Thu Feb 22 05:59:25 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 22 Feb 2018 10:59:25 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia Message-ID: https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Meets_Julia_Micro_Performance?lang=en -- Steve From bc at freeuk.com Thu Feb 22 07:03:09 2018 From: bc at freeuk.com (bartc) Date: Thu, 22 Feb 2018 12:03:09 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 22/02/2018 10:59, Steven D'Aprano wrote: > https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Meets_Julia_Micro_Performance?lang=en While an interesting article on speed-up techniques, that seems to miss the point of benchmarks. On the fib(20) test, it suggests using this to get a 30,000 times speed-up: from functools import lru_cache as cache @cache(maxsize=None) def fib_cache(n): if n<2: return n return fib_cache(n-1)+fib_cache(n-2) The idea of the Fibonacci benchmark is to test how effectively an implementation manages large numbers of recursive function calls. Then, fib(36) would normally involve 48,315,633 calls. This version does only 37, giving a misleading impression. Anyway, I got a 6x speed-up using pypy, without changing anything. Although, I doubt if that's still executing actual byte-code, if /that/ was the point of the test. (It then goes on to suggest using 'numba', and using its JIT compiler, and using on that on an /iterative/ version of fib(). Way to miss the point. It might be a technique to bear in mind, but it is nonsensical to say this gives a 17,000 times speed-up over the original code. Here's another speed-up I found myself, although it was only 50 times faster, not 17,000: just write the code in C, and call it via os.system("fib.exe"). But you /do/ need to write it in a different language.) -- bartc From rhodri at kynesim.co.uk Thu Feb 22 07:09:30 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 22 Feb 2018 12:09:30 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: <403522a9-cff6-1e78-37bd-abae63b63be7@kynesim.co.uk> On 22/02/18 10:59, Steven D'Aprano wrote: > https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Meets_Julia_Micro_Performance?lang=en Interesting article. I can't help but feel that using Cython is cheating a bit, and I was really expecting a bit more Pythonic rewriting of the benchmarks Pythonic, but still food for thought. -- Rhodri James *-* Kynesim Ltd From rosuav at gmail.com Thu Feb 22 07:29:53 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 22 Feb 2018 23:29:53 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On Thu, Feb 22, 2018 at 11:03 PM, bartc wrote: > On 22/02/2018 10:59, Steven D'Aprano wrote: >> >> >> https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Meets_Julia_Micro_Performance?lang=en > > > While an interesting article on speed-up techniques, that seems to miss the > point of benchmarks. > > On the fib(20) test, it suggests using this to get a 30,000 times speed-up: > > from functools import lru_cache as cache > > @cache(maxsize=None) > def fib_cache(n): > if n<2: > return n > return fib_cache(n-1)+fib_cache(n-2) > > The idea of the Fibonacci benchmark is to test how effectively an > implementation manages large numbers of recursive function calls. Then, > fib(36) would normally involve 48,315,633 calls. > > This version does only 37, giving a misleading impression. Not overly misleading; the point of it is to show how trivially easy it is to memoize a function in Python. For a fair comparison, I'd like to see the equivalent Julia code: the function, unchanged, with something around the outside of it to manage caching and memoization. Can that be done with a couple of trivial lines of code using only the standard library? ChrisA From none at gmail.com Thu Feb 22 07:32:45 2018 From: none at gmail.com (ast) Date: Thu, 22 Feb 2018 13:32:45 +0100 Subject: File opening modes (r, w, a ...) Message-ID: <5a8eb870$0$31641$426a74cc@news.free.fr> Hello I share a very valuable table I found on StackOverflow about file opening modes If like me you always forget the details of file opening mode, the following table provides a good summary | r r+ w w+ a a+ ------------------|-------------------------- read | + + + + write | + + + + + write after seek | + + + create | + + + + truncate | + + position at start | + + + + position at end | + + From storchaka at gmail.com Thu Feb 22 08:51:39 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 22 Feb 2018 15:51:39 +0200 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: 22.02.18 14:29, Chris Angelico ????: > Not overly misleading; the point of it is to show how trivially easy > it is to memoize a function in Python. For a fair comparison, I'd like > to see the equivalent Julia code: the function, unchanged, with > something around the outside of it to manage caching and memoization. > Can that be done with a couple of trivial lines of code using only the > standard library? The article contains a reference to a Julia example. > Note that Julia also can memoize functions, see this example [1] provided by Ismael V.C. [1] http://nbviewer.jupyter.org/gist/Ismael-VC/b04ad17ee44c89917803 From breamoreboy at gmail.com Thu Feb 22 09:02:46 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Thu, 22 Feb 2018 14:02:46 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 22/02/18 12:03, bartc wrote: > > It might be a technique to bear in mind, but it is nonsensical to say > this gives a 17,000 times speed-up over the original code. What makes you say that? I have just run all of the code as given in the reference and get results that are in the same ball park for every test that was run. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From tkadm30 at yandex.com Thu Feb 22 09:02:54 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Thu, 22 Feb 2018 09:02:54 -0500 Subject: Django-hotsauce 0.9.1 release Message-ID: <360afe69-123c-d87d-fbd2-99f8db26eb8a@yandex.com> Hi everyone, I'm glad to announce the release of Django-hotsauce 0.9.1: https://www.isotopesoftware.ca/pub/django-hotsauce/django-hotsauce-0.9.1.tar.gz This is primarly a bugfix release for PyPy. Users of django-hotsauce are encouraged to test this release (with PyPy or Python 3) and submit bug reports here: https://bitbucket.org/tkadm30/django-hotsauce/issues For any questions or comments related to this project, please join the mailing list: https://groups.google.com/forum/#!forum/django-hotsauce Best regards, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From larry.martell at gmail.com Thu Feb 22 10:06:00 2018 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 22 Feb 2018 10:06:00 -0500 Subject: atws In-Reply-To: References: Message-ID: On Thu, Feb 22, 2018 at 2:00 AM, Chris Angelico wrote: > On Thu, Feb 22, 2018 at 5:27 PM, Larry Martell wrote: >> I want to use the atws package >> (https://atws.readthedocs.io/readme.html). I am using python 2.7.6 on >> ubuntu-trusty-64 3.13.0-87-generic. I get this error when importing >> the package: >> >>>>> import atws >> Traceback (most recent call last): >> File "", line 1, in >> File "/usr/local/lib/python2.7/dist-packages/atws/__init__.py", line >> 4, in >> from .wrapper import connect >> File "/usr/local/lib/python2.7/dist-packages/atws/wrapper.py", line >> 32, in >> from . import connection >> File "/usr/local/lib/python2.7/dist-packages/atws/connection.py", >> line 19, in >> from requests.exceptions import ConnectTimeout, Timeout, >> ReadTimeout, SSLError >> ImportError: cannot import name ConnectTimeout >> >> I would not be surprised if no one here has used this package, but has >> anyone seen this error in other packages or their own work? > > You're running that on a fairly old version of Python (2.7.6 - the > latest in the 2.x branch is 2.7.14). It's entirely possible that the > app doesn't support the versions of Python and/or the 'requests' > library that you have installed. > > First thing to try: can you use the app under Python 3? This is a mature django app and I cannot switch to Python 3. > For reference, here's the version of requests that I have (which does > have that exception available): > >>>> import requests >>>> requests.__version__ > '2.18.4' > > What's yours? I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I get: No handlers could be found for logger "atws.connection" From info at wingware.com Thu Feb 22 10:09:31 2018 From: info at wingware.com (Wingware) Date: Thu, 22 Feb 2018 10:09:31 -0500 Subject: ANN: Wing Python IDEs version 6.0.10 released Message-ID: <5A8EDD2B.3090300@wingware.com> Hi, We've just released Wing 6.0.10 , which adds support for Python 3.7 and the new breakpoint() builtin, improves reliability of remote development through network breaks, improves display of names for threads started with the threading module, fixes detecting the Python interpreter for Nuke , adds support for cygwin Python 3.6, and makes about 40 other improvements. For details, see https://wingware.com/pub/wingide/6.0.10/CHANGELOG.txt Download Now About Wing Wing is a family of cross-platform Python IDEs with powerful integrated editing, debugging, unit testing, and project management features. Wing runs on Windows, Linux, and OS X, and can be used to develop any kind of Python code for web, desktop, embedded scripting, and other applications. Wing 101 and Wing Personal omit some features and are free to download and use without a license. Wing Pro requires purchasing or upgrading a license, or obtaining a 30-day trial at startup. Version 6 introduces many new features, including improved multi-selection, much easier remote development , debugging from the Python Shell, recursive debugging, PEP 484 and 526 type hinting, support for Python 3.6 and 3.7, Vagrant , Jupyter , and Django 1.10+, easier Raspberry Pi development, optimized debugger, OS X full screen mode, One Dark color palette, expanded free product line, and much more. For details, see What's New in Wing Version 6 . Wing 6 works with Python versions 2.5 through 2.7 and 3.2 through 3.7, including also Anaconda, ActivePython, EPD, Stackless, and others derived from the CPython implementation. For more product information, please visit wingware.com Upgrading You can try Wing 6 without removing older versions. Wing 6 will read and convert your old preferences, settings, and projects. Projects should be saved to a new name since previous versions of Wing cannot read Wing 6 projects. See also Migrating from Older Versions and Upgrading . Links Release notice: https://wingware.com/news/2018-02-13 Downloads and Free Trial: https://wingware.com/downloads Buy: https://wingware.com/store/purchase Upgrade: https://wingware.com/store/upgrade Questions? Don't hesitate to email us at support at wingware.com. Thanks, -- Stephan Deibel Wingware | Python IDE The Intelligent Development Environment for Python Programmers wingware.com From none at gmail.com Thu Feb 22 10:15:54 2018 From: none at gmail.com (ast) Date: Thu, 22 Feb 2018 16:15:54 +0100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: <5a8edead$0$20427$426a74cc@news.free.fr> Le 22/02/2018 ? 13:03, bartc a ?crit?: > On 22/02/2018 10:59, Steven D'Aprano wrote: >> https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Meets_Julia_Micro_Performance?lang=en >> > > While an interesting article on speed-up techniques, that seems to miss > the point of benchmarks. > > On the fib(20) test, it suggests using this to get a 30,000 times speed-up: > > ??? from functools import lru_cache as cache > > ??? @cache(maxsize=None) > ??? def fib_cache(n): > ??????? if n<2: > ??????????? return n > ??????? return fib_cache(n-1)+fib_cache(n-2) > It's a meaningless to test the execution time of a function with a cache decorator on 1.000.000 loops The first execution, ok, you get something meaningfull but for the other 999.999 executions, the result is already on the cache so you just measure the time to read the result in a dictionnary and output it. On my computer: >>> setup = """\ from functools import lru_cache as cache @cache(maxsize=None) def fib(n): if n < 2: return n return fib(n-1) + fib(n-2) """ >>> from timeit import timeit >>> timeit("fib(20)", setup=setup, number=1) 0.00010329007704967808 >>> timeit("fib(20)", setup=setup, number=100) 0.0001489834564836201 so 100 loops or 1 loop provides similar results as expected ! From breamoreboy at gmail.com Thu Feb 22 10:25:23 2018 From: breamoreboy at gmail.com (Mark Lawrence) Date: Thu, 22 Feb 2018 15:25:23 +0000 Subject: atws In-Reply-To: References: Message-ID: On 22/02/18 15:06, Larry Martell wrote: > On Thu, Feb 22, 2018 at 2:00 AM, Chris Angelico wrote: >> On Thu, Feb 22, 2018 at 5:27 PM, Larry Martell wrote: >>> I want to use the atws package >>> (https://atws.readthedocs.io/readme.html). I am using python 2.7.6 on >>> ubuntu-trusty-64 3.13.0-87-generic. I get this error when importing >>> the package: >>> >>>>>> import atws >>> Traceback (most recent call last): >>> File "", line 1, in >>> File "/usr/local/lib/python2.7/dist-packages/atws/__init__.py", line >>> 4, in >>> from .wrapper import connect >>> File "/usr/local/lib/python2.7/dist-packages/atws/wrapper.py", line >>> 32, in >>> from . import connection >>> File "/usr/local/lib/python2.7/dist-packages/atws/connection.py", >>> line 19, in >>> from requests.exceptions import ConnectTimeout, Timeout, >>> ReadTimeout, SSLError >>> ImportError: cannot import name ConnectTimeout >>> >>> I would not be surprised if no one here has used this package, but has >>> anyone seen this error in other packages or their own work? >> >> You're running that on a fairly old version of Python (2.7.6 - the >> latest in the 2.x branch is 2.7.14). It's entirely possible that the >> app doesn't support the versions of Python and/or the 'requests' >> library that you have installed. >> >> First thing to try: can you use the app under Python 3? > > This is a mature django app and I cannot switch to Python 3. > >> For reference, here's the version of requests that I have (which does >> have that exception available): >> >>>>> import requests >>>>> requests.__version__ >> '2.18.4' >> >> What's yours? > > I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I get: > > No handlers could be found for logger "atws.connection" > I get exactly the same thing, but do the import interactively and then help(atws) shows help so I'm guessing that it's just a poor informational message, so I suggest that you just try running your code and see what happens. p.s. enjoying the curling at the Winter Olympics? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From bc at freeuk.com Thu Feb 22 11:00:35 2018 From: bc at freeuk.com (bartc) Date: Thu, 22 Feb 2018 16:00:35 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 22/02/2018 12:03, bartc wrote: > On the fib(20) test, it suggests using this to get a 30,000 times speed-up: BTW while doing my tests, I found you could redefine the same function with no error: def fred(): pass def fred(): pass def fred(): pass For classes too. I was aware you could reassign a different value to such names, but didn't know this applied to def. I guess the reason is that 'def' is just another statement, and statements can be conditional, so that you can have several versions. It was still a surprise. What an easy way for things to go wrong by writing a new version of a function but forgetting to delete the original. And if they happen to be in the right order so the program still works, it must be confusing to read. -- bartc From steve+comp.lang.python at pearwood.info Thu Feb 22 11:00:48 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 22 Feb 2018 16:00:48 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Thu, 22 Feb 2018 12:03:09 +0000, bartc wrote: > The idea of the Fibonacci benchmark is to test how effectively an > implementation manages large numbers of recursive function calls. Then, > fib(36) would normally involve 48,315,633 calls. > > This version does only 37, giving a misleading impression. Who cares if it only does 37? There is *absolutely no benefit* to those additional 48,315,596 calls, and thanks to the benchmark I discovered that.[1] I want to know what is the fastest implementation of Fibonacci I can write. I'm not married to the idea that I have to make a ludicrous 48 million function calls just to get the 36th Fibonacci number. > (It then goes on to suggest using 'numba', and using its JIT compiler, > and using on that on an /iterative/ version of fib(). Way to miss the > point. Indeed, you certainly have. > It might be a technique to bear in mind, but it is nonsensical to say > this gives a 17,000 times speed-up over the original code. That's *exactly what it did*. How many years, decades, have you been programming? Have you not realised yet that the best way to optimize code is to pick the fastest algorithm you can that will do the job? There's no law that says because you started with a slow, inefficient algorithm you have to stay with it. > Here's another speed-up I found myself, although it was only 50 times > faster, not 17,000: just write the code in C, and call it via > os.system("fib.exe"). Did you include the time taken to capture the text output from stdout, parse it, and convert to an int? Did your C code support numbers greater than 2**64? My fibonacci function can calculate the 10,000th Fibonacci number in a blink of the eye: py> fibi(10000) 336447648764317832666216120051075...976171121233066073310059947366875 Output truncated for brevity, it is a 2090-digit number. Admittedly it did take about a minute and a half to generate all 208988 digits of the one millionth Fibonacci number, but the performance is fast enough for my needs. [1] Actually I already knew it, but then I didn't perform these benchmarks, I'm just linking to them. -- Steve From steve+comp.lang.python at pearwood.info Thu Feb 22 11:03:20 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 22 Feb 2018 16:03:20 +0000 (UTC) Subject: File opening modes (r, w, a ...) References: <5a8eb870$0$31641$426a74cc@news.free.fr> Message-ID: On Thu, 22 Feb 2018 13:32:45 +0100, ast wrote: > Hello > > I share a very valuable table I found on StackOverflow about file > opening modes > > If like me you always forget the details of file opening mode, the > following table provides a good summary Thanks! -- Steve From ned at nedbatchelder.com Thu Feb 22 11:55:43 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 22 Feb 2018 11:55:43 -0500 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: <79b2f51e-983c-9eed-1c9d-2e54454cf7cf@nedbatchelder.com> On 2/22/18 11:00 AM, bartc wrote: > On 22/02/2018 12:03, bartc wrote: > >> On the fib(20) test, it suggests using this to get a 30,000 times >> speed-up: > > BTW while doing my tests, I found you could redefine the same function > with no error: > > def fred(): > ????pass > > def fred(): > ????pass > > def fred(): > ????pass > > For classes too. I was aware you could reassign a different value to > such names, but didn't know this applied to def. > > I guess the reason is that 'def' is just another statement, and > statements can be conditional, so that you can have several versions. More importantly, "def x" is just a convenient way to express an assignment to x.? It behaves exactly like "x = make_a_function(name='x', ...)" (where I'm waving my hands about what happens in the ellipsis... :)?? Just as there is no prohibition on "x = 1; x = 2", there's no prohibition on "def x(); def x()" Lots of Python statements are assignments to names, with precisely the same referencing and scoping behaviors as conventional assignment. All of these assign to x: ??? x = ... ??? def x(...) ??? def fn(x) ??? class x ??? import x ??? import blah as x ??? from blah import x ??? for x in ... ??? with blah as x ??? except Something as x --Ned. From tkadm30 at yandex.com Thu Feb 22 12:00:45 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Thu, 22 Feb 2018 12:00:45 -0500 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: <03aa3036-b0a6-694d-bd51-e40722c10471@yandex.com> So, how exactly can PyPy and JIT runs multithreaded Python applications any faster than Julia on distributed systems? Right now I think PyPy and JIT can run Python code on my old ia32 computer faster than with Python/Cython. How do Julia scale on x86 machines ? Etienne Le 2018-02-22 ? 11:00, Steven D'Aprano a ?crit?: > On Thu, 22 Feb 2018 12:03:09 +0000, bartc wrote: > >> The idea of the Fibonacci benchmark is to test how effectively an >> implementation manages large numbers of recursive function calls. Then, >> fib(36) would normally involve 48,315,633 calls. >> >> This version does only 37, giving a misleading impression. > Who cares if it only does 37? There is *absolutely no benefit* to those > additional 48,315,596 calls, and thanks to the benchmark I discovered > that.[1] > > I want to know what is the fastest implementation of Fibonacci I can > write. I'm not married to the idea that I have to make a ludicrous 48 > million function calls just to get the 36th Fibonacci number. > > >> (It then goes on to suggest using 'numba', and using its JIT compiler, >> and using on that on an /iterative/ version of fib(). Way to miss the >> point. > Indeed, you certainly have. > > >> It might be a technique to bear in mind, but it is nonsensical to say >> this gives a 17,000 times speed-up over the original code. > That's *exactly what it did*. > > How many years, decades, have you been programming? Have you not realised > yet that the best way to optimize code is to pick the fastest algorithm > you can that will do the job? There's no law that says because you > started with a slow, inefficient algorithm you have to stay with it. > > >> Here's another speed-up I found myself, although it was only 50 times >> faster, not 17,000: just write the code in C, and call it via >> os.system("fib.exe"). > Did you include the time taken to capture the text output from stdout, > parse it, and convert to an int? > > Did your C code support numbers greater than 2**64? My fibonacci function > can calculate the 10,000th Fibonacci number in a blink of the eye: > > py> fibi(10000) > 336447648764317832666216120051075...976171121233066073310059947366875 > > Output truncated for brevity, it is a 2090-digit number. > > Admittedly it did take about a minute and a half to generate all 208988 > digits of the one millionth Fibonacci number, but the performance is fast > enough for my needs. > > > > > [1] Actually I already knew it, but then I didn't perform these > benchmarks, I'm just linking to them. > > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From larry.martell at gmail.com Thu Feb 22 12:24:06 2018 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 22 Feb 2018 12:24:06 -0500 Subject: atws In-Reply-To: References: Message-ID: On Thu, Feb 22, 2018 at 10:25 AM, Mark Lawrence wrote: > On 22/02/18 15:06, Larry Martell wrote: >> >> On Thu, Feb 22, 2018 at 2:00 AM, Chris Angelico wrote: >>> >>> On Thu, Feb 22, 2018 at 5:27 PM, Larry Martell >>> wrote: >>>> >>>> I want to use the atws package >>>> (https://atws.readthedocs.io/readme.html). I am using python 2.7.6 on >>>> ubuntu-trusty-64 3.13.0-87-generic. I get this error when importing >>>> the package: >>>> >>>>>>> import atws >>>> >>>> Traceback (most recent call last): >>>> File "", line 1, in >>>> File "/usr/local/lib/python2.7/dist-packages/atws/__init__.py", line >>>> 4, in >>>> from .wrapper import connect >>>> File "/usr/local/lib/python2.7/dist-packages/atws/wrapper.py", line >>>> 32, in >>>> from . import connection >>>> File "/usr/local/lib/python2.7/dist-packages/atws/connection.py", >>>> line 19, in >>>> from requests.exceptions import ConnectTimeout, Timeout, >>>> ReadTimeout, SSLError >>>> ImportError: cannot import name ConnectTimeout >>>> >>>> I would not be surprised if no one here has used this package, but has >>>> anyone seen this error in other packages or their own work? >>> >>> >>> You're running that on a fairly old version of Python (2.7.6 - the >>> latest in the 2.x branch is 2.7.14). It's entirely possible that the >>> app doesn't support the versions of Python and/or the 'requests' >>> library that you have installed. >>> >>> First thing to try: can you use the app under Python 3? >> >> >> This is a mature django app and I cannot switch to Python 3. >> >>> For reference, here's the version of requests that I have (which does >>> have that exception available): >>> >>>>>> import requests >>>>>> requests.__version__ >>> >>> '2.18.4' >>> >>> What's yours? >> >> >> I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I >> get: >> >> No handlers could be found for logger "atws.connection" >> > > I get exactly the same thing, but do the import interactively and then > help(atws) shows help so I'm guessing that it's just a poor informational > message, so I suggest that you just try running your code and see what > happens. Thanks - it seems to work despite that message. > p.s. enjoying the curling at the Winter Olympics? For sure - huge upset over Canada! I stayed up until 3am watching the women's hockey game last night and I had to be up at 6 this morning, so I'm dragging now. From bc at freeuk.com Thu Feb 22 12:53:30 2018 From: bc at freeuk.com (bartc) Date: Thu, 22 Feb 2018 17:53:30 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 22/02/2018 16:00, Steven D'Aprano wrote: > On Thu, 22 Feb 2018 12:03:09 +0000, bartc wrote: > >> The idea of the Fibonacci benchmark is to test how effectively an >> implementation manages large numbers of recursive function calls. Then, >> fib(36) would normally involve 48,315,633 calls. >> >> This version does only 37, giving a misleading impression. > > Who cares if it only does 37? There is *absolutely no benefit* to those > additional 48,315,596 calls, and thanks to the benchmark I discovered > that.[1] > I want to know what is the fastest implementation of Fibonacci I can > write. I'm not married to the idea that I have to make a ludicrous 48 > million function calls just to get the 36th Fibonacci number. As I said, people keep missing the point. The fact this uses a grossly inefficient way of calculating Fibonacci seems to blind them to any other considerations. The actual result is irrelevant, so long as its correct. The important thing is those 50 million calls. If you want to know how well one language implementations deals with function calls compared to another, you will prefer a test that is doing 48 million function calls and little else, rather than one that does only three dozen. And one that is harder for a compiler to optimise, as you want to know how fast those 48 million calls can be made, not how well a compiler can avoid doing them! >> (It then goes on to suggest using 'numba', and using its JIT compiler, >> and using on that on an /iterative/ version of fib(). Way to miss the >> point. > > Indeed, you certainly have. > > >> It might be a technique to bear in mind, but it is nonsensical to say >> this gives a 17,000 times speed-up over the original code. > > That's *exactly what it did*. By running a different program? Fine, then here's my submission for the N=20 case used in the tests: def fib(n): return 6765 You have to compare like with like when testing languages, otherwise there's no point. If you want to use an iterative algorithm, then do the same with Julia. But then these functions are so fast, that it's not clear exactly what you are testing. > Did your C code support numbers greater than 2**64? My fibonacci function > can calculate the 10,000th Fibonacci number in a blink of the eye: > > py> fibi(10000) > 336447648764317832666216120051075...976171121233066073310059947366875 > > Output truncated for brevity, it is a 2090-digit number. > > Admittedly it did take about a minute and a half to generate all 208988 > digits of the one millionth Fibonacci number, but the performance is fast > enough for my needs. (OK, my interpreter took nearly 4 minutes, but I use my own big integer algorithms. All jolly interesting, but a diversion.) The fact is that the vast majority of integer calculations don't need to use big integers (pretty much 100% of mine). Probably most don't even need 64 bits, but 32 bits. So, if there is a performance advantage in having separate 64-bit and big integer types, then why not make use of it? There's been a lot of talk of the advantages of static typing, and here is one use for it. -- Bartc From rosuav at gmail.com Thu Feb 22 13:40:43 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Feb 2018 05:40:43 +1100 Subject: atws In-Reply-To: References: Message-ID: On Fri, Feb 23, 2018 at 2:06 AM, Larry Martell wrote: > On Thu, Feb 22, 2018 at 2:00 AM, Chris Angelico wrote: >> For reference, here's the version of requests that I have (which does >> have that exception available): >> >>>>> import requests >>>>> requests.__version__ >> '2.18.4' >> >> What's yours? > > I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I get: > > No handlers could be found for logger "atws.connection" Cool! Sounds like it's all working then. Yay for backward compatibility in requests, so it didn't demand lots of other upgrades :) ChrisA From rosuav at gmail.com Thu Feb 22 13:43:16 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Feb 2018 05:43:16 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On Fri, Feb 23, 2018 at 12:51 AM, Serhiy Storchaka wrote: > 22.02.18 14:29, Chris Angelico ????: >> >> Not overly misleading; the point of it is to show how trivially easy >> it is to memoize a function in Python. For a fair comparison, I'd like >> to see the equivalent Julia code: the function, unchanged, with >> something around the outside of it to manage caching and memoization. >> Can that be done with a couple of trivial lines of code using only the >> standard library? > > > The article contains a reference to a Julia example. > >> Note that Julia also can memoize functions, see this example [1] provided >> by Ismael V.C. > [1] http://nbviewer.jupyter.org/gist/Ismael-VC/b04ad17ee44c89917803 > Yep, but I don't know Julia enough to know if that's the standard library, and there aren't timings for it (which also makes it harder to skim for - I missed it on first read, and only found it when I searched the page for 'memoize'). If that's standard library, my conclusion would be "Winner: Tie". ChrisA From rosuav at gmail.com Thu Feb 22 13:53:26 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Feb 2018 05:53:26 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <5a8edead$0$20427$426a74cc@news.free.fr> References: <5a8edead$0$20427$426a74cc@news.free.fr> Message-ID: On Fri, Feb 23, 2018 at 2:15 AM, ast wrote: > Le 22/02/2018 ? 13:03, bartc a ?crit : >> >> On 22/02/2018 10:59, Steven D'Aprano wrote: >>> >>> >>> https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Meets_Julia_Micro_Performance?lang=en >> >> >> While an interesting article on speed-up techniques, that seems to miss >> the point of benchmarks. >> >> On the fib(20) test, it suggests using this to get a 30,000 times >> speed-up: >> >> from functools import lru_cache as cache >> >> @cache(maxsize=None) >> def fib_cache(n): >> if n<2: >> return n >> return fib_cache(n-1)+fib_cache(n-2) >> > > It's a meaningless to test the execution time of a function > with a cache decorator on 1.000.000 loops > > The first execution, ok, you get something meaningfull > but for the other 999.999 executions, the result is already on > the cache so you just measure the time to read the result > in a dictionnary and output it. > > On my computer: > >>>> setup = """\ > from functools import lru_cache as cache > @cache(maxsize=None) > def fib(n): > if n < 2: return n > return fib(n-1) + fib(n-2) > """ >>>> from timeit import timeit > >>>> timeit("fib(20)", setup=setup, number=1) > 0.00010329007704967808 > >>>> timeit("fib(20)", setup=setup, number=100) > 0.0001489834564836201 > > so 100 loops or 1 loop provides similar results > as expected ! > The solution would be to flush the cache in the core code. Here's a tweak: from timeit import timeit setup = """ from functools import lru_cache as cache def fib(n): if n < 2: return n return fib(n-1) + fib(n-2) # this is what we effectively do inside the loop: @cache(maxsize=None) def fib_cached(n): return fib(n) """ for count in 1, 10, 100, 1000: print(count, timeit("cache(maxsize=None)(fib)(20)", setup=setup, number=count)) You could improve on performance by keeping the same function object and just flushing the cache with fib.cache_clear(), but I have no idea how you'd translate that into other languages. Constructing a cache at run-time should be safe. ChrisA From rosuav at gmail.com Thu Feb 22 14:07:44 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Feb 2018 06:07:44 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On Fri, Feb 23, 2018 at 3:00 AM, Steven D'Aprano wrote: > On Thu, 22 Feb 2018 12:03:09 +0000, bartc wrote: >> Here's another speed-up I found myself, although it was only 50 times >> faster, not 17,000: just write the code in C, and call it via >> os.system("fib.exe"). > > Did you include the time taken to capture the text output from stdout, > parse it, and convert to an int? Relatively trivial compared to the overhead of invoking a subprocess THROUGH A SHELL. On Windows, that has to invoke cmd.exe; on Unix-like systems, most likely /bin/sh. And then that has to search the system path (maybe that doesn't happen on his Windows system, as presumably it's finding the executable on the cwd check). That's file system operations, and a lot of them. > Did your C code support numbers greater than 2**64? My fibonacci function > can calculate the 10,000th Fibonacci number in a blink of the eye: > > py> fibi(10000) > 336447648764317832666216120051075...976171121233066073310059947366875 > > Output truncated for brevity, it is a 2090-digit number. > > Admittedly it did take about a minute and a half to generate all 208988 > digits of the one millionth Fibonacci number, but the performance is fast > enough for my needs. Yeah, but if you stick that into a variable, you can separately time the calculation from the stringification. Using the exact code from the page: def fib_seq(n): if n < 2: return n a,b = 1,0 for i in range(n-1): a,b = a+b,a return a >>> time.time(); x = fib_seq(1000000); time.time() 1519325946.7880309 1519325953.773382 That's seven seconds to calculate the number. How big a number is it? >>> x.bit_length() 694241 >>> math.log10(x) 208987.29076497658 Both take effectively zero time. Dividing that number by 10 that many times is what takes all the processing. (Stringifying a number basically consists of div-mod to trim off the last digit, keep going until you run out of number. Lots of division.) So actually calculating the millionth Fibonacci number can be done fairly quickly; figuring out its first digits would be hard, but we don't actually need that. Though we could probably get a decent estimate: >>> log = math.log10(x) >>> print(10**(log - int(log)), "e +", int(log)) 1.9532821287892859 e + 208987 I wouldn't trust all of those digits, but that's a fast way to get an idea of the scale of a number. Doing that rather than printing the whole number means you actually see the performance of *calculation*. ChrisA From jack at alcor.concordia.ca Thu Feb 22 14:55:08 2018 From: jack at alcor.concordia.ca (Jack Fearnley) Date: Thu, 22 Feb 2018 19:55:08 -0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Fri, 23 Feb 2018 06:07:44 +1100, Chris Angelico wrote: > On Fri, Feb 23, 2018 at 3:00 AM, Steven D'Aprano > wrote: >> On Thu, 22 Feb 2018 12:03:09 +0000, bartc wrote: >>> Here's another speed-up I found myself, although it was only 50 times >>> faster, not 17,000: just write the code in C, and call it via >>> os.system("fib.exe"). >> >> Did you include the time taken to capture the text output from stdout, >> parse it, and convert to an int? > > Relatively trivial compared to the overhead of invoking a subprocess > THROUGH A SHELL. On Windows, that has to invoke cmd.exe; on Unix-like > systems, most likely /bin/sh. And then that has to search the system > path (maybe that doesn't happen on his Windows system, as presumably > it's finding the executable on the cwd check). That's file system > operations, and a lot of them. > >> Did your C code support numbers greater than 2**64? My fibonacci >> function can calculate the 10,000th Fibonacci number in a blink of the >> eye: >> >> py> fibi(10000) >> 336447648764317832666216120051075...976171121233066073310059947366875 >> >> Output truncated for brevity, it is a 2090-digit number. >> >> Admittedly it did take about a minute and a half to generate all 208988 >> digits of the one millionth Fibonacci number, but the performance is >> fast enough for my needs. > > Yeah, but if you stick that into a variable, you can separately time the > calculation from the stringification. Using the exact code from the > page: > > def fib_seq(n): > if n < 2: > return n > a,b = 1,0 for i in range(n-1): > a,b = a+b,a > return a > >>>> time.time(); x = fib_seq(1000000); time.time() > 1519325946.7880309 1519325953.773382 > > That's seven seconds to calculate the number. How big a number is it? > >>>> x.bit_length() > 694241 >>>> math.log10(x) > 208987.29076497658 > > Both take effectively zero time. Dividing that number by 10 that many > times is what takes all the processing. (Stringifying a number basically > consists of div-mod to trim off the last digit, keep going until you run > out of number. Lots of division.) > > So actually calculating the millionth Fibonacci number can be done > fairly quickly; figuring out its first digits would be hard, but we > don't actually need that. Though we could probably get a decent > estimate: > >>>> log = math.log10(x) >>>> print(10**(log - int(log)), "e +", int(log)) > 1.9532821287892859 e + 208987 > > I wouldn't trust all of those digits, but that's a fast way to get an > idea of the scale of a number. Doing that rather than printing the whole > number means you actually see the performance of *calculation*. I realize that this thread is about benchmarking and not really about generating fibonacci numbers, but I hope nobody is using this code to generate them on a 'production' basis, Fibonacci numbers, any linearly recursive sequence for that matter, can be generated in log time. GP/Pari on my Intel I7 computes fibonacci(100000) in less than 1 ms, fibonacci(1000000) in 5ms, fibonacci(10000000) in 59 ms and finally fibonacci(100000000) in 733 ms. This would obviously be slower in Python but by a constant factor. Best regards, Jack Fearnley > > ChrisA From none at gmail.com Thu Feb 22 15:15:12 2018 From: none at gmail.com (ast) Date: Thu, 22 Feb 2018 21:15:12 +0100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <5a8edead$0$20427$426a74cc@news.free.fr> Message-ID: <5a8f24d2$0$3701$426a34cc@news.free.fr> Le 22/02/2018 ? 19:53, Chris Angelico a ?crit?: > On Fri, Feb 23, 2018 at 2:15 AM, ast wrote: >> Le 22/02/2018 ? 13:03, bartc a ?crit : >>> >>> On 22/02/2018 10:59, Steven D'Aprano wrote: >>>> >>>> > for count in 1, 10, 100, 1000: > print(count, timeit("cache(maxsize=None)(fib)(20)", setup=setup, > number=count)) hum ... very astute From bc at freeuk.com Thu Feb 22 16:05:36 2018 From: bc at freeuk.com (bartc) Date: Thu, 22 Feb 2018 21:05:36 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 22/02/2018 19:55, Jack Fearnley wrote: > I realize that this thread is about benchmarking and not really about > generating fibonacci numbers, but I hope nobody is using this code to > generate them on a 'production' basis, > > Fibonacci numbers, any linearly recursive sequence for that matter, can > be generated in log time. > > GP/Pari on my Intel I7 computes fibonacci(100000) in less than 1 ms, > fibonacci(1000000) in 5ms, The simple method involves 1 million additions of numbers with an average length of 100,000 digits. 5ms would be pretty good going. Presumably it uses a faster algorithm. I found this in Python (from stackoverflow): def fib(n): v1, v2, v3 = 1, 1, 0 # initialise a matrix [[1,1],[1,0]] print (bin(n)[3:]) for rec in bin(n)[3:]: # perform fast exponentiation of the .... calc = v2*v2 v1, v2, v3 = v1*v1+calc, (v1+v3)*v2, calc+v3*v3 if rec=='1': v1, v2, v3 = v1+v2, v1, v2 return v2 fib(1000000) took 200ms seconds in Python 3. Printing the result about another 1.6 seconds. I think now it's down to the efficiency of the big integer library, as little bytecode is being executed. -- bartc From ian.g.kelly at gmail.com Thu Feb 22 17:28:01 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 22 Feb 2018 15:28:01 -0700 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On Thu, Feb 22, 2018 at 12:55 PM, Jack Fearnley wrote: > I realize that this thread is about benchmarking and not really about > generating fibonacci numbers, but I hope nobody is using this code to > generate them on a 'production' basis, > > Fibonacci numbers, any linearly recursive sequence for that matter, can > be generated in log time. To be pedantic, that's not really true; see below. > GP/Pari on my Intel I7 computes fibonacci(100000) in less than 1 ms, > fibonacci(1000000) in 5ms, fibonacci(10000000) in 59 ms and finally > fibonacci(100000000) in 733 ms. I'm not sure exactly what algorithm that's using, but note that the growth of these timings is *not* logarithmic. In fact, it's superlinear. It's true can generate Fibonacci numbers in log(n) iterations using linear algebra. As long as multiplication and addition are assumed to be constant time, then the Fibonacci algorithm is indeed O(log n) time. However, the numbers in the sequence grow on the order of O(phi ** n) which quickly exceeds what you can fit into a 32-bit integer. Therefore bigints are quickly required. The fastest known integer multiplication algorithm is O(n * log n * log log n). The Fibonacci numbers grow at O(phi ** n) and therefore require O(log (phi ** n)) = O(n) bits to represent them. Each iteration therefore requires O(n * log n * log log n) time, and with O(log n) iterations the total time of the algorithm is thus O(n * (log n) ** 2 * log log n). It's still a lot faster than the brute force approach, and aeons faster than the brute-force recursive approach, but it's not strictly logarithmic because that part of the analysis is dwarfed by the cost of the arithmetic involved. From bc at freeuk.com Thu Feb 22 19:26:33 2018 From: bc at freeuk.com (bartc) Date: Fri, 23 Feb 2018 00:26:33 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 22/02/2018 14:02, Mark Lawrence wrote: > On 22/02/18 12:03, bartc wrote: > >> >> It might be a technique to bear in mind, but it is nonsensical to say >> this gives a 17,000 times speed-up over the original code. > > What makes you say that?? I have just run all of the code as given in > the reference and get results that are in the same ball park for every > test that was run. It's nonsensical because you are comparing two very different algorithms, especially when one could well be a million times faster than the other. The point of the article was Julia vs. Python. You can't make Python faster by switching to a faster algorithm; you have to use the same one on both. -- bartc From steve+comp.lang.python at pearwood.info Thu Feb 22 19:31:49 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 00:31:49 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Thu, 22 Feb 2018 19:55:08 +0000, Jack Fearnley wrote: > I realize that this thread is about benchmarking and not really about > generating fibonacci numbers, but I hope nobody is using this code to > generate them on a 'production' basis, > > Fibonacci numbers, any linearly recursive sequence for that matter, can > be generated in log time. That is not what your timing results show. Your timing results are *worse* than linear, not better. In fact, they look more like N log N: N Your time ? N ? log N ? N log N (-actual-) (----------predicted---------) 100000 <1? 0.5 4.2 0.4 1000000 5 5 5.0 5.0 10000000 59 50 5.8 58.3 100000000 733 500 6.7 666.7 1000000000 ? 5000 7.5 7500.0 I don't know what algorithm your version of Fibonacci is using, but if it is the one which uses matrix multiplication, then it is logarithmic in the number of *steps*, but each step doesn't take constant time. As the numbers involved get larger, the cost of each multiplication and addition becomes higher. Big O analysis is never a substitute for actual timing measurements, and the assumption behind Big O analysis, namely that only some operations take time, and always constant time, is never correct. It is only an approximation to the truth, and as you can see, something which is algorithmically Big O logarithmic can turn out to scale worse than linearly once you actually measure the times. -- Steve From rosuav at gmail.com Thu Feb 22 19:53:57 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Feb 2018 11:53:57 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On Fri, Feb 23, 2018 at 11:31 AM, Steven D'Aprano wrote: > Big O analysis is never a substitute for actual timing measurements, and > the assumption behind Big O analysis, namely that only some operations > take time, and always constant time, is never correct. It is only an > approximation to the truth, and as you can see, something which is > algorithmically Big O logarithmic can turn out to scale worse than > linearly once you actually measure the times. I prefer to describe that as "hidden costs". Any step that has its own non-constant cost becomes a hidden cost. That doesn't mean Big O is itself limited, but that when you analyze a function, you have to think about more than just what's right in front of you. For example: def count_at_max(items): count = 0 for item in items: if item == max(items): count += 1 return count def count_at_max(items): count = 0 peak = max(items) for item in items: if item == peak: count += 1 return count So similar, but one of them runs in O(n?) and the other in O(n). The max() call is a "hidden cost". (Obviously they'll usually be subtler than that, but this is a sledgehammer for demonstrative purposes.) Big O *can* handle this, you just have to take notice of things. ChrisA From rantingrickjohnson at gmail.com Thu Feb 22 20:13:02 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Thu, 22 Feb 2018 17:13:02 -0800 (PST) Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: <3cf440d6-ca55-40c1-9bc7-a58ade413bbb@googlegroups.com> On Thursday, February 22, 2018 at 1:55:35 PM UTC-6, Jack Fearnley wrote: [...] > I realize that this thread is about benchmarking and not > really about generating fibonacci numbers, but I hope > nobody is using this code to generate them on a > 'production' basis, I've been raising the warning flags about that awful Fibonacci example for years! First it was in the tutorial, and then, they plastered it on the front page of Python.org like it were some sort of trophy... At this point i have concluded that it must be: (1) a sadistic noob hazing device, or (2) a straw-stuffed, anthropomorphic, crow-poop-riddled monster that sends the seasoned programmers off in the opposite direction. From steve+comp.lang.python at pearwood.info Thu Feb 22 20:27:53 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 01:27:53 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Thu, 22 Feb 2018 17:53:30 +0000, bartc wrote: > On 22/02/2018 16:00, Steven D'Aprano wrote: >> On Thu, 22 Feb 2018 12:03:09 +0000, bartc wrote: >> >>> The idea of the Fibonacci benchmark is to test how effectively an >>> implementation manages large numbers of recursive function calls. >>> Then, fib(36) would normally involve 48,315,633 calls. >>> >>> This version does only 37, giving a misleading impression. >> >> Who cares if it only does 37? There is *absolutely no benefit* to those >> additional 48,315,596 calls, and thanks to the benchmark I discovered >> that.[1] > >> I want to know what is the fastest implementation of Fibonacci I can >> write. I'm not married to the idea that I have to make a ludicrous 48 >> million function calls just to get the 36th Fibonacci number. > > As I said, people keep missing the point. The fact this uses a grossly > inefficient way of calculating Fibonacci seems to blind them to any > other considerations. > > The actual result is irrelevant, so long as its correct. The important > thing is those 50 million calls. Why do you care about the 50 million calls? That's crazy -- the important thing is *calculating the Fibonacci numbers as efficiently as possible*. I don't believe for a second that you would write code like this: # need a Fibonacci number, but the implementation we have # is too efficient, only taking 50 calls instead of 50 million # so call the function a million times for i in range(1000000): num = fibonacci(40) do_something_useful(num) so why on earth do you care about the 50 million calls? The only reason I care about them is to reduce the number as much as possible. Why do you care about them? > If you want to know how well one language implementations deals with > function calls compared to another, you will prefer a test that is doing > 48 million function calls and little else, rather than one that does > only three dozen. I don't care about "how well one language implementations deals with function calls compared to another". I care about writing the fastest idiomatic code I can in the language I'm using. If I were writing Julia code, I would write the best Julia code I could. If I were writing Python code, I would write the best Python code I can. If I'm writing C or Pascal, that would mean writing loops like this: arr := some array for i := 1 to len(arr): value = arr[i] process(value) If I'm writing Ruby or Python, that means *NOT* writing loops like that, but writing them like this: for value in arr: process(value) If the Julia coders have decided, for whatever reason, that they want to show off how super-efficient function recursion is in Julia by using a doubly-recursive implementation of the Fibonacci sequence, I'm not bound by their choice. Their motive is to highlight the advantages of Julia. My motive is to show how Python can be just as fast. When beginners complain that "Python is too slow", nine times out of ten they are writing non-idiomatic, inefficient code. Python is not Java: http://dirtsimple.org/2004/12/python-is-not-java.html nor is it Julia. > And one that is harder for a compiler to optimise, as you want to know > how fast those 48 million calls can be made, not how well a compiler can > avoid doing them! I don't give a flying fox about how fast the compiler can do those 48 million calls, I want to know how to get Fibonacci numbers as fast as I can, and if that means avoiding making the calls, then you better believe that I will avoid making those calls. Why are you benchmarking code you would never use in real code? I would be a really shitty programmer if I insisted on writing Julia code in Python just because the Julia developers write Julia code in Julia. Just as I would be a shitty programmer if I insisted that Julia programmers must write *Python* code in Julia. Write code that is natural and efficient for the language you are using. Otherwise you end up making really ludicrous comparisons between languages that have *zero* relationship to real-world code that anyone in their right mind would actually use. > By running a different program? Fine, then here's my submission for the > N=20 case used in the tests: > > def fib(n): return 6765 The program has to be *correct*, and your program is not. When comparing two compilers, you are ALWAYS comparing two different programs. Either the compilers generate precisely the same machine/byte code, in which case there is no point in comparing them[1], or they generate different machine/byte code, in which case you're comparing two different running programs. [...] > The fact is that the vast majority of integer calculations don't need to > use big integers (pretty much 100% of mine). Probably most don't even > need 64 bits, but 32 bits. And here we have the World According To Bart again: "since *I* don't need more than 32-bits, then obviously it is a FACT than nobody else could need them". Yes Bart, the reason that so many languages have support for Bignums is that everyone except you is an idiot who loves writing slow code for absolutely no reason at all. [1] Except to verify that they are, in fact, the same. -- Steve From jfong at ms4.hinet.net Thu Feb 22 20:28:06 2018 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Thu, 22 Feb 2018 17:28:06 -0800 (PST) Subject: File opening modes (r, w, a ...) In-Reply-To: <5a8eb870$0$31641$426a74cc@news.free.fr> References: <5a8eb870$0$31641$426a74cc@news.free.fr> Message-ID: ast? 2018?2?22???? UTC+8??8?33?00???? > Hello > > I share a very valuable table I found on > StackOverflow about file opening modes > > If like me you always forget the details of > file opening mode, the following table provides > a good summary > > | r r+ w w+ a a+ > ------------------|-------------------------- > read | + + + + > write | + + + + + > write after seek | + + + > create | + + + + > truncate | + + > position at start | + + + + > position at end | + + What the "write after seek" means? --Jach From python at bladeshadow.org Thu Feb 22 20:36:21 2018 From: python at bladeshadow.org (Python) Date: Thu, 22 Feb 2018 19:36:21 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: <20180223013621.GA10273@bladeshadow.org> On Thu, Feb 22, 2018 at 11:29:53PM +1100, Chris Angelico wrote: > > The idea of the Fibonacci benchmark is to test how effectively an > > implementation manages large numbers of recursive function calls. Then, > > fib(36) would normally involve 48,315,633 calls. > > > > This version does only 37, giving a misleading impression. > > Not overly misleading; the point of it is to show how trivially easy > it is to memoize a function in Python. This does not appear to me at all to be the point of the article. The point of the article seems to be that the Julia benchmarks compare unfairly the performance of Python to Julia, because they do not use algorithms that suit "the best way to use Python." But I have to agree with bartc, that is not at all the point of the benchmarks. The point of the benchmarks is to compare the performance of a particular algorithm or set of algorithms across a variety of languages. It's fine to say that this method of computing Fibonacci sequences is inefficient; but anyone who's spent any time on the problem knows that recursive function calls is not the most efficient way to calculate them in most languages. So it must follow logically that the point of the benchmark is not to prove that Julia is faster than Python at solving Fibonacci sequences, but rather that Julia is faster than Python at solving the class of problems that would be implemented in a similar manner as this particular implementation of Fibonacci sequences. No other interpretation makes any sense. And if that is the point, as logic dictates, then comparing the performance of algorithm A implemented in language X to the performance of a completely different algorithm B in language Y is generally both misleading and pointless. If you can prove that A is the most efficient alogorithm possible in X and B is the most efficient algorithm in Y, for solving the same problem, then and only then, you might have something. Otherwise, I think what you're saying is bunk. From lew.pitcher at digitalfreehold.ca Thu Feb 22 20:42:59 2018 From: lew.pitcher at digitalfreehold.ca (Lew Pitcher) Date: Thu, 22 Feb 2018 20:42:59 -0500 Subject: File opening modes (r, w, a ...) References: <5a8eb870$0$31641$426a74cc@news.free.fr> Message-ID: jfong at ms4.hinet.net wrote: > ast? 2018?2?22???? UTC+8??8?33?00???? >> Hello >> >> I share a very valuable table I found on >> StackOverflow about file opening modes >> >> If like me you always forget the details of >> file opening mode, the following table provides >> a good summary >> >> | r r+ w w+ a a+ >> ------------------|-------------------------- >> read | + + + + >> write | + + + + + >> write after seek | + + + >> create | + + + + >> truncate | + + >> position at start | + + + + >> position at end | + + > > What the "write after seek" means? It /should/ mean that programs are permitted to seek to a point in the file, and then write from that point on. A write to a read mode ("r") file isn't permitted at all, so neither is "write after seek" to a read mode file. A write to an append mode ("a" and "a+") file always write to the end of the file, effectively negating any seek. HTH -- Lew Pitcher "In Skills, We Trust" PGP public key available upon request From python at bladeshadow.org Thu Feb 22 22:31:28 2018 From: python at bladeshadow.org (Python) Date: Thu, 22 Feb 2018 21:31:28 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: <20180223033128.GB10273@bladeshadow.org> On Fri, Feb 23, 2018 at 01:27:53AM +0000, Steven D'Aprano wrote: > On Thu, 22 Feb 2018 17:53:30 +0000, bartc wrote: > > As I said, people keep missing the point. The fact this uses a grossly > > inefficient way of calculating Fibonacci seems to blind them to any > > other considerations. > > > > The actual result is irrelevant, so long as its correct. The important > > thing is those 50 million calls. > > Why do you care about the 50 million calls? That's crazy -- the important > thing is *calculating the Fibonacci numbers as efficiently as possible*. If you are writing practical programs, that's true. But the Julia benchmarks are not practical programs; they are designed to compare the performance of various language features across a range of languages. How is it a fair comparison to compare how Julia runs an algorithm that is not optimal in pretty much any language, to a completely different one with a much improved order of complexity, optimized specifically for Python? The article author even points out that the Fibonacci sequence problem can be solved using the same technique as he used for his Python solution, but makes no effort to benchmark it and then goes on to declare Python the winner anyway. That's just bad science. Many of the comments after the article repeatedly echo the same fundamental flaw that bartc raised: The benchmark isn't intended to determine the fastest way to calculate Fibonacci numbers; it's meant to measure the language's efficiency solving this class of problems in this way. > I don't give a flying fox about how fast the compiler can do those 48 > million calls And if you needed to write a program that HAD TO perform those 48 million calls, wouldn't you care then? Don't forget that in the real world, such reasons can include things like, "I don't know any other way to solve it and I need it done RFN," or, "My boss is a moron and insisted I do it this way..." [If you've never seen the latter, consider yourself lucky, but I sadly have.] > When comparing two compilers, you are ALWAYS comparing two different > programs. Such benchmarks still compare the efficiency with which the compiler turns the same algorithm you wrote in its provided syntax (ultimately) into executed instructions, and that's still a fair and useful comparison of the two languages, for solutions to problems of that particular class/structure/whatever. Knowing that a particular problem has a more efficient solution in the language you're using isn't always feasible; no one's knowledge of algorithms is exhaustive, and most of us have to care about deadlines way more than whether or not our implementations are exactly the most efficient ones possible. The average programmer typically has very little control or even awareness of what code the compiler generates. You can say that makes them shitty programmers but guess what? The world is full of shitty programmers. In that world, the real world, if you forbid these types of comparisons for being inherently unfair, simply because the compiler will obviously not generate *precisely* the same code, then you can not with any measure of sanity determine which language is the better tool to solve your problem of the day, given the tools you have at your disposal (including your own knowledge of algorithms). These types of benchmarks are useful for discovering what a given language is good at and what it is not. That information is useful to either allow you to choose the more efficient language for the solution you're going to implement, or perhaps even as a hint that the language you want to use may have a more efficient way to achieve the result you need. From jfong at ms4.hinet.net Thu Feb 22 23:15:49 2018 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Thu, 22 Feb 2018 20:15:49 -0800 (PST) Subject: File opening modes (r, w, a ...) In-Reply-To: References: <5a8eb870$0$31641$426a74cc@news.free.fr> Message-ID: <21a0c2bb-ef47-457f-b04e-517d8a59d76c@googlegroups.com> Lew Pitcher? 2018?2?23???? UTC+8??9?43?19???? > jfong at ms4.hinet.net wrote: > > > ast? 2018?2?22???? UTC+8??8?33?00???? > >> Hello > >> > >> I share a very valuable table I found on > >> StackOverflow about file opening modes > >> > >> If like me you always forget the details of > >> file opening mode, the following table provides > >> a good summary > >> > >> | r r+ w w+ a a+ > >> ------------------|-------------------------- > >> read | + + + + > >> write | + + + + + > >> write after seek | + + + > >> create | + + + + > >> truncate | + + > >> position at start | + + + + > >> position at end | + + > > > > What the "write after seek" means? > > It /should/ mean that programs are permitted to seek to a point in the file, > and then write from that point on. > > A write to a read mode ("r") file isn't permitted at all, > so neither is "write after seek" to a read mode file. > > A write to an append mode ("a" and "a+") file always write to the end of the > file, effectively negating any seek. > > HTH > > -- > Lew Pitcher > "In Skills, We Trust" > PGP public key available upon request Thank you for explanation. Mode 'r+', 'a' and 'a+' all can be seek and write, but only 'r+' was marked in the table. That's why I was confused. This row seems redundant to me:-) --Jach From steve+comp.lang.python at pearwood.info Fri Feb 23 00:00:13 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 05:00:13 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Fri, 23 Feb 2018 00:26:33 +0000, bartc wrote: > The point of the article was Julia vs. Python. You can't make Python > faster by switching to a faster algorithm; you have to use the same one > on both. No, the point of the article was to write Python code that is as fast as the Julia code. I don't know why the Julia programmers chose to use such a poor algorithm: maybe they are show-casing just how amazingly awesome the Julia compiler is that it can even make a rubbish algorithm run fast. I don't care -- they can choose whatever algorithm they like for their Julia code, but I don't really care how good the language is at running bad code that I would never use in real life. -- Steve From tjreedy at udel.edu Fri Feb 23 01:38:01 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Feb 2018 01:38:01 -0500 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <20180223013621.GA10273@bladeshadow.org> References: <20180223013621.GA10273@bladeshadow.org> Message-ID: On 2/22/2018 8:36 PM, Python wrote: > On Thu, Feb 22, 2018 at 11:29:53PM +1100, Chris Angelico wrote: >>> The idea of the Fibonacci benchmark is to test how effectively an >>> implementation manages large numbers of recursive function calls. Then, >>> fib(36) would normally involve 48,315,633 calls. >>> >>> This version does only 37, giving a misleading impression. >> >> Not overly misleading; the point of it is to show how trivially easy >> it is to memoize a function in Python. > > This does not appear to me at all to be the point of the article. The > point of the article seems to be that the Julia benchmarks compare > unfairly the performance of Python to Julia, because they do not use > algorithms that suit "the best way to use Python." But I have to > agree with bartc, that is not at all the point of the benchmarks. The > point of the benchmarks is to compare the performance of a particular > algorithm or set of algorithms across a variety of languages. > > It's fine to say that this method of computing Fibonacci sequences is > inefficient; but anyone who's spent any time on the problem knows that > recursive function calls is not the most efficient way to calculate > them in most languages. So it must follow logically that the point of > the benchmark is not to prove that Julia is faster than Python at > solving Fibonacci sequences, but rather that Julia is faster than > Python at solving the class of problems that would be implemented in a > similar manner as this particular implementation of Fibonacci > sequences. It is no secret that Python's interpreted function calls are slower than function calls compiled to machine code and that Python's infinite precision integer arithmetic is slower that machine int arithmetic. So there is almost no point in combining both in a naive drastically inefficient algorithm and declaring that Python is slower. As to the vague 'class of problems implemented in a similar manner': Any function f of count N that depends of values of f for counts < N can be memoized the same way in Python as fibonacci. Everything said about P vs J for fib applies to such similar problems. The same is almost true for functions that depend on a pair of counts, such as the number of combinations of N things taken K at a time. The time benefit per space used is just a bit less. Now consider a general recursive problem: a graph with N nodes and E edges and recursive node functions that depend on the value of the function at args(node) other nodes. Example: the minimum length of a path from node a to node b. In the field of graph algorithms, it is completely routine to save f(node) for intermediate nodes when calculated. The idea of naively implementing the recursive definition, in any non-trivial practical situation, without saving intermediate values, is pretty ludicrous. Fib can be viewed as a function on a directed graph where all but the 2 base nodes have two incoming and two outgoing edges. -- Terry Jan Reedy From dieter at handshake.de Fri Feb 23 02:01:39 2018 From: dieter at handshake.de (dieter) Date: Fri, 23 Feb 2018 08:01:39 +0100 Subject: atws References: Message-ID: <871shcfauk.fsf@handshake.de> Larry Martell writes: > ... > I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I get: > > No handlers could be found for logger "atws.connection" This is a warning (only), telling you that the "atws" package wants to log a message but there is not corresponding logging handler defined -- likely because you did not configure Python's logging infrastructure (--> "logging" package in Python's standard documentation). From tjreedy at udel.edu Fri Feb 23 03:11:36 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Feb 2018 03:11:36 -0500 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <20180223033128.GB10273@bladeshadow.org> References: <20180223033128.GB10273@bladeshadow.org> Message-ID: On 2/22/2018 10:31 PM, Python wrote: >> Why do you care about the 50 million calls? That's crazy -- the important >> thing is *calculating the Fibonacci numbers as efficiently as possible*. > If you are writing practical programs, that's true. But the Julia > benchmarks are not practical programs; they are designed to compare > the performance of various language features across a range of > languages. If that were so, then the comparison should use the fastest *Python* implementation. Which is what the article discussed here did. But the comparison is a lie when the comparison is compiled machine code versus bytecode interpreted by crippled cpython*. And people use this sort of benchmark to choose a language for *practical programs*, and don't know that they are not seeing *Python* times, but *crippled cpython* times. * Python has an import statement. But 'comparisons' disallow 'import numpy', a quite legal Python statement, and similar others. The ability to import Python wrappers of Fortran and C libraries is part of the fundamental design of cpython. It is a distributed project. The fact that numerical python, numarray, and now numpy are distributed separately, in spite of proposals to include them, syntax additions for their benefit, and their overwhelming usefullness, is a matter of social and administrative convenience, not necessity. There is also a proposal, which I consider likely to happen, to enhance the cpython Windows and Mac installers by making installation of numpy and other great 'external' modules selectable options. It takes just 2 or 3 legal Python lines that do run with cpython as installed to make 'import numpy' work, without resorting to subprocess (though that is *also* legal python: from ensurepip import bootstrap; bootstrap() # only if needed from pip import main main(['install', 'numpy']) >>> import numpy >>> dir(numpy) ['ALLOW_THREADS', 'AxisError', 'BUFSIZE', 'CLIP', 'ComplexWarning', ... 'vstack', 'warnings', 'where', 'who', 'zeros', 'zeros_like'] So, given a realistic number-crunching benchmark that take at least several minutes, one could add those two lines at the top and be off -- Terry Jan Reedy From rosuav at gmail.com Fri Feb 23 03:58:01 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Feb 2018 19:58:01 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223013621.GA10273@bladeshadow.org> Message-ID: On Fri, Feb 23, 2018 at 5:38 PM, Terry Reedy wrote: > As to the vague 'class of problems implemented in a similar manner': Any > function f of count N that depends of values of f for counts < N can be > memoized the same way in Python as fibonacci. Everything said about P vs J > for fib applies to such similar problems. The same is almost true for > functions that depend on a pair of counts, such as the number of > combinations of N things taken K at a time. The time benefit per space used > is just a bit less. The naive recursive Fibonacci function can be memoized with maxsize=2 and it gets virtually all the benefit - in fact, it becomes effectively iterative. (I think even maxsize=1 will do that.) With most memoization, that kind of benefit is less blatant. ChrisA From alister.ware at ntlworld.com Fri Feb 23 04:19:41 2018 From: alister.ware at ntlworld.com (alister) Date: Fri, 23 Feb 2018 09:19:41 GMT Subject: How to make Python run as fast (or faster) than Julia References: <20180223033128.GB10273@bladeshadow.org> Message-ID: On Fri, 23 Feb 2018 03:11:36 -0500, Terry Reedy wrote: > On 2/22/2018 10:31 PM, Python wrote: > >>> Why do you care about the 50 million calls? That's crazy -- the >>> important thing is *calculating the Fibonacci numbers as efficiently >>> as possible*. > >> If you are writing practical programs, that's true. But the Julia >> benchmarks are not practical programs; they are designed to compare the >> performance of various language features across a range of languages. > > If that were so, then the comparison should use the fastest *Python* > implementation. Which is what the article discussed here did. But the > comparison is a lie when the comparison is compiled machine code versus > bytecode interpreted by crippled cpython*. And people use this sort of > benchmark to choose a language for *practical programs*, and don't know > that they are not seeing *Python* times, but *crippled cpython* times. Benchmarks are completely pointless anyway. the question when choosing a language should not be 'How fast it is', it should always be 'Is it fast enough'. once you have a choice of languages that are fast enough other criteria for suitability become important (such as readability & ease of debugging) -- If you're going to do something tonight that you'll be sorry for tomorrow morning, sleep late. -- Henny Youngman From stefan_ml at behnel.de Fri Feb 23 05:00:28 2018 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 23 Feb 2018 11:00:28 +0100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: Steven D'Aprano schrieb am 22.02.2018 um 11:59: > https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Meets_Julia_Micro_Performance?lang=en Thanks for sharing, Steven. While it was already suggested between the lines in some of the replies, I'd like to emphasise that the combination of timeit and result caching (memoizing) in this post is misleading and not meaningful. It actually shows a common mistake that easily happens when benchmarking code. Since timeit repeats the benchmark runs and takes the minimum, it will *always* return the time it takes to look up the final result in the cache, and never report the actual performance of the code that is meant to be benchmarked here. From what I read, this was probably not intended by the author of the post. Myself, I'm guilty as charged to run into this more than once and have seen it in many occasions. I've even seen proper Cython benchmark code that a C compiler can fully analyse as static and replaces by a constant, and then get wonder speedups from it that would never translate to any real-world gains. These things happen, but they are mistakes. All they tell us is that we must always be careful when evaluating benchmarks and their results, and to take good care that the benchmarks match the intended need, which is to help us understand and solve a real-world performance problem [1]. Stefan [1] That's also the problem in the post above and in the original benchmarks it refers to: there is no real-world problem to be solved here. Someone wrote slow code and then showed that one language can evaluate that slow code faster than another. Nothing to see here, keep walking... From steve+comp.lang.python at pearwood.info Fri Feb 23 05:20:35 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 10:20:35 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Fri, 23 Feb 2018 11:00:28 +0100, Stefan Behnel wrote: > I've even seen proper Cython benchmark code that a C compiler can fully > analyse as static and replaces by a constant, and then get wonder > speedups from it that would never translate to any real-world gains. This is one of the reasons why typical benchmarks are junk. If you're not benchmarking actual, real-world code you care about, you have no idea how the language will perform with actual, real-world code. There is a popular meme that "Python is slow" based on the results of poorly-written code. And yet, in *real world code* that people actually use, Python's performance is not that bad -- especially when you play to its strengths, rather than its weaknesses, and treat it as a glue language for stitching together high-performance libraries written in high-performance languages like C, Fortran, Rust and Java. Speaking of bad benchmarks, I recall an anecdote about a programmer running some benchmarks on Fortran code on various mainframes, and found that one specific machine was millions of times faster than the others. It turns out that the code being tested was something like this (translated into Python): for i in range(1, 1000000): pass and the Fortan compiler on that one machine was clever enough to treat it as dead code and remove it, turning the entire benchmark into a single no-op. Your point that we must consider benchmarkers carefully is a good one. -- Steve From steve+comp.lang.python at pearwood.info Fri Feb 23 05:21:07 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 10:21:07 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Fri, 23 Feb 2018 11:00:28 +0100, Stefan Behnel wrote: > I've even seen proper Cython benchmark code that a C compiler can fully > analyse as static and replaces by a constant, and then get wonder > speedups from it that would never translate to any real-world gains. This is one of the reasons why typical benchmarks are junk. If you're not benchmarking actual, real-world code you care about, you have no idea how the language will perform with actual, real-world code. There is a popular meme that "Python is slow" based on the results of poorly-written code. And yet, in *real world code* that people actually use, Python's performance is not that bad -- especially when you play to its strengths, rather than its weaknesses, and treat it as a glue language for stitching together high-performance libraries written in high-performance languages like C, Fortran, Rust and Java. Speaking of bad benchmarks, I recall an anecdote about a programmer running some benchmarks on Fortran code on various mainframes, and found that one specific machine was millions of times faster than the others. It turns out that the code being tested was something like this (translated into Python): for i in range(1, 1000000): pass and the Fortan compiler on that one machine was clever enough to treat it as dead code and remove it, turning the entire benchmark into a single no-op. Your point that we must consider benchmarkers carefully is a good one. -- Steve From bc at freeuk.com Fri Feb 23 07:17:50 2018 From: bc at freeuk.com (bartc) Date: Fri, 23 Feb 2018 12:17:50 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 23/02/2018 01:27, Steven D'Aprano wrote: > On Thu, 22 Feb 2018 17:53:30 +0000, bartc wrote: >> The actual result is irrelevant, so long as its correct. The important >> thing is those 50 million calls. > > Why do you care about the 50 million calls? Because we are interested in comparing call-efficiency across languages? NOT in Fibonacci numbers. That's crazy -- the important > thing is *calculating the Fibonacci numbers as efficiently as possible*. > > I don't believe for a second that you would write code like this: > > > # need a Fibonacci number, but the implementation we have We don't. If Fibonacci causes problems for you because you can't see past the inefficiency, then we might use the Ackermann function instead. (But I've found Ackermann can cause stack overflow in some languages.) > so why on earth do you care about the 50 million calls? > > The only reason I care about them is to reduce the number as much as > possible. Why do you care about them? You're responsible for implementing the call-mechanism of a new language, and you want to make it efficient. What program do you test it with, one that contains zero calls (clearly, that will be most efficient!), or one that contains lots of calls? > arr := some array > for i := 1 to len(arr): > value = arr[i] > process(value) > > > If I'm writing Ruby or Python, that means *NOT* writing loops like that, > but writing them like this: > > for value in arr: > process(value) Suppose it turned out that in Python, /this/ was actually faster: for i in range(len(arr)): value = arr[i] process(value) ? > I don't give a flying fox about how fast the compiler can do those 48 > million calls, I want to know how to get Fibonacci numbers as fast as I > can, and if that means avoiding making the calls, then you better believe > that I will avoid making those calls. OK, then, if you are at all interested in Julia vs. Python (and it was you who posted that article!), then compare a version of a benchmark that uses as few calls as possible, if you like. But what exactly is that benchmark then highlighting? Loops? >> By running a different program? Fine, then here's my submission for the >> N=20 case used in the tests: >> >> def fib(n): return 6765 > > The program has to be *correct*, and your program is not. OK, then do as I did, and keep a global tally of how many calls it makes, and print that as the result. Then the correct output for 'fibcalls(36)' should be: 48315633 If yours shows only 37, then /your/ program is wrong. (BTW, this value will always fit in 64 bits, for the foreseeable future.) > When comparing two compilers, you are ALWAYS comparing two different > programs. That's part of the art of doing it properly. > > Either the compilers generate precisely the same machine/byte code, in > which case there is no point in comparing them[1], or they generate > different machine/byte code, in which case you're comparing two different > running programs. So the people who worked for years on PyPy never once bothered to compare results with running the same source program with CPython, because there would be point? OK, I never knew that... >> The fact is that the vast majority of integer calculations don't need to >> use big integers (pretty much 100% of mine). Probably most don't even >> need 64 bits, but 32 bits. > > And here we have the World According To Bart again: "since *I* don't need > more than 32-bits, then obviously it is a FACT than nobody else could > need them". Do you have better statistics than me? If so please share. I guess the number of calculations (that /can/ be done in 64 bits) is less than 100%, but not 0%, so somewhere in between. I'm saying it's nearer to 100% than to 0%; you're saying, what ..... ? As examples of quantities that can be represented in 64 bits: Character codes or code points Integer pixel values Most enumerations For-loop counting (if it needs more than 64 bits, then we might be waiting a while for it to finish) Array indexing Numbers of people, cars, guns, countries, houses, books, iphones etc on Earth ... endless examples of numbers less than 9000000000000000000 used in programming Most calculations on these can also be done in 64 bits (except multiply and power where the ceiling might 32 bits or lower). > Yes Bart, the reason that so many languages have support for Bignums is > that everyone except you is an idiot who loves writing slow code for > absolutely no reason at all. If bignum support is a handicap, even when they are not needed, then it is something to consider. -- bartc From rosuav at gmail.com Fri Feb 23 07:41:44 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Feb 2018 23:41:44 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On Fri, Feb 23, 2018 at 11:17 PM, bartc wrote: >>> The fact is that the vast majority of integer calculations don't need to >>> use big integers (pretty much 100% of mine). Probably most don't even >>> need 64 bits, but 32 bits. >> >> >> And here we have the World According To Bart again: "since *I* don't need >> more than 32-bits, then obviously it is a FACT than nobody else could >> need them". > > > Do you have better statistics than me? If so please share. I guess the > number of calculations (that /can/ be done in 64 bits) is less than 100%, > but not 0%, so somewhere in between. > > I'm saying it's nearer to 100% than to 0%; you're saying, what ..... ? I don't know what he's saying, but I'm saying that unless it is 100%, your language should just natively support bigints. How do you know when you'll need that extra room? Are you going to pick bigints for everything that's affected by user input? If so, you may as well use them for everything and not bother with the distinction; if not, some day, your program will need larger numbers than you guessed for it, and it'll break. > As examples of quantities that can be represented in 64 bits: > > Character codes or code points Sure. That's why we have a special optimization for sequences of 21-bit numbers. We call it "str". > Integer pixel values Maybe in 64 bits for the time being, but 32 certainly won't be enough. As soon as you do any sort of high DPI image manipulation, you will exceed 2**32 total pixels in an image (that's just 65536x65536, or 46341x46341 if you're using signed integers); and it wouldn't surprise me if some image manipulation needs that many on a single side - if not today, then tomorrow. So 64 bits might not be enough once you start counting total pixels. > Most enumerations > For-loop counting (if it needs more than 64 bits, then we might be > waiting a while for it to finish) Most loops don't need any counting. Should we implement a 0-bit integer for those? > Array indexing > Numbers of people, cars, guns, countries, houses, books, iphones etc > on Earth > ... endless examples of numbers less than 9000000000000000000 used > in programming As soon as you say "the number of X is always going to be less than Y", you open yourself up to risk. https://xkcd.com/865/ > Most calculations on these can also be done in 64 bits (except multiply and > power where the ceiling might 32 bits or lower). Exactly. Most. And then you'll run into some limitation somewhere. Let's say you want to know how many IP addresses, on average, are allocated to one person. You'll probably get a single digit number as your answer. Can you calculate that using 32-bit integers? What about 64-bit? Should you use bignums just in case? Take your pick, let me know, and then I'll tell you my answer - and why. >> Yes Bart, the reason that so many languages have support for Bignums is >> that everyone except you is an idiot who loves writing slow code for >> absolutely no reason at all. > > If bignum support is a handicap, even when they are not needed, then it is > something to consider. What sort of handicap is it? Performance? That's what optimization is for. Some languages (Pike, for instance) have a single data type that is sometimes represented internally by a native integer and sometimes by a bignum; it's completely transparent apart from with timings. Others (eg Python 2) have two separate data types, but will smoothly transition to bignum any time it's needed. IMO CPython 3.x could probably benefit from the transparent optimization... but Python (the language) benefits hugely from the core int type simply being a bignum. There are no semantically significant effects at the boundary between "small numbers" and "large numbers". Auto-growing lists are a cost, too. It's expensive to have to allocate more memory to allow another element to be added. Should we lock in the sizes of all lists/arrays at creation? Why should we pay the price for that flexibility? Or maybe, in Bartville, this flexibility is worth paying for but large integers are useless? ChrisA From bc at freeuk.com Fri Feb 23 07:57:35 2018 From: bc at freeuk.com (bartc) Date: Fri, 23 Feb 2018 12:57:35 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> Message-ID: On 23/02/2018 08:11, Terry Reedy wrote: > * Python has an import statement.? But 'comparisons' disallow 'import > numpy', a quite legal Python statement, and similar others. If I'm duplicating a benchmark [in another language] then the last thing I want to see is something like 'import numpy', or other special language feature, as that cannot be replicated. I want to see a pure algorithm. Not the equivalent of: os.system("C_version.exe") ? The ability > to import Python wrappers of Fortran and C libraries is part of the > fundamental design of cpython.? It is a distributed project. Is numpy a general purpose C library that can also be called from any language that can use a C API? Or is it specific to Python? -- bartc From rosuav at gmail.com Fri Feb 23 08:03:06 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 00:03:06 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> Message-ID: On Fri, Feb 23, 2018 at 11:57 PM, bartc wrote: > On 23/02/2018 08:11, Terry Reedy wrote: > >> * Python has an import statement. But 'comparisons' disallow 'import >> numpy', a quite legal Python statement, and similar others. > > > If I'm duplicating a benchmark [in another language] then the last thing I > want to see is something like 'import numpy', or other special language > feature, as that cannot be replicated. > > I want to see a pure algorithm. Not the equivalent of: > > os.system("C_version.exe") > > The ability >> >> to import Python wrappers of Fortran and C libraries is part of the >> fundamental design of cpython. It is a distributed project. > > > Is numpy a general purpose C library that can also be called from any > language that can use a C API? Or is it specific to Python? > No, it's a general purpose FORTRAN library that can also be called from any language that can use a generic C, FORTRAN, COBOL, etc API. ChrisA From larry.martell at gmail.com Fri Feb 23 08:08:58 2018 From: larry.martell at gmail.com (Larry Martell) Date: Fri, 23 Feb 2018 08:08:58 -0500 Subject: atws In-Reply-To: <871shcfauk.fsf@handshake.de> References: <871shcfauk.fsf@handshake.de> Message-ID: On Fri, Feb 23, 2018 at 2:01 AM, dieter wrote: > Larry Martell writes: >> ... >> I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I get: >> >> No handlers could be found for logger "atws.connection" > > This is a warning (only), telling you that the "atws" package wants > to log a message but there is not corresponding logging handler > defined -- likely because you did not configure Python's logging > infrastructure (--> "logging" package in Python's standard documentation). Yes, but this message comes from the import, which happens before any of my code runs. From tkadm30 at yandex.com Fri Feb 23 08:16:13 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 23 Feb 2018 08:16:13 -0500 Subject: do_handshake_on_connect should not be specified for non-blocking sockets Message-ID: <6ed025a9-61ed-7467-1f80-afc69457772d@yandex.com> How can i fix this error? I'm using gevent/uwsgi and python 2.7.13... Looks like ssl.py really needs monkey-patching to work with gevent/uwsgi, but i'm not sure exactly where/what triggers this error. Any thoughts? Etienne File "/home/www/isotoperesearch.ca/trunk/www-bin/dispatch-django.uwsgi", line 17, in application ??? return wsgi_app(environ, start_response) ? File "/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/middleware.py", line 228, in __call__ ??? code) ? File "/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/client.py", line 114, in request_access_token ??? data=urlencode(form).encode('utf-8')) ? File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen ??? return opener.open(url, data, timeout) ? File "/usr/lib/python2.7/urllib2.py", line 429, in open ??? response = self._open(req, data) ? File "/usr/lib/python2.7/urllib2.py", line 447, in _open ??? '_open', req) ? File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain ??? result = func(*args) ? File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open ??? context=self._context) ? File "/usr/lib/python2.7/urllib2.py", line 1195, in do_open ??? h.request(req.get_method(), req.get_selector(), req.data, headers) ? File "/usr/lib/python2.7/httplib.py", line 1042, in request ??? self._send_request(method, url, body, headers) ? File "/usr/lib/python2.7/httplib.py", line 1082, in _send_request ??? self.endheaders(body) ? File "/usr/lib/python2.7/httplib.py", line 1038, in endheaders ??? self._send_output(message_body) ? File "/usr/lib/python2.7/httplib.py", line 882, in _send_output ??? self.send(msg) ? File "/usr/lib/python2.7/httplib.py", line 844, in send ??? self.connect() ? File "/usr/lib/python2.7/httplib.py", line 1263, in connect ??? server_hostname=server_hostname) ? File "/usr/lib/python2.7/ssl.py", line 363, in wrap_socket ??? _context=self) ? File "/usr/lib/python2.7/ssl.py", line 610, in __init__ ??? raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets") ValueError: do_handshake_on_connect should not be specified for non-blocking sockets From bc at freeuk.com Fri Feb 23 08:28:16 2018 From: bc at freeuk.com (bartc) Date: Fri, 23 Feb 2018 13:28:16 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 23/02/2018 12:41, Chris Angelico wrote: > On Fri, Feb 23, 2018 at 11:17 PM, bartc wrote: >> Integer pixel values > > Maybe in 64 bits for the time being, but 32 certainly won't be enough. Why, people's eyes will evolve to see quintillions of colours? > As soon as you do any sort of high DPI image manipulation, you will > exceed 2**32 total pixels in an image (that's just 65536x65536, or > 46341x46341 if you're using signed integers) (Huh?) ; and it wouldn't surprise > me if some image manipulation needs that many on a single side - if > not today, then tomorrow. So 64 bits might not be enough once you > start counting total pixels. A four-billion x four-billion image would be quite something. I wonder how long it would take Cpython to loop through all 18e18 pixels? (BTW this still fits into unsigned 64 bits.) These speculations are pointless. Graphics is very hardware-oriented now, and this is mainly the concern of graphics processors which will already have wide datapaths, much wider than 64 bits. Because you can of course use more than one 64-bit value. >> Most calculations on these can also be done in 64 bits (except multiply and >> power where the ceiling might 32 bits or lower). > > Exactly. Most. OK, good, we've agreed on something: Most. Actually most values in my programs fit into 32 bits. Most of /those/ are likely to fit into 16 bits. In a program, most integers are going to represent small values. And then you'll run into some limitation somewhere. > > Let's say you want to know how many IP addresses, on average, are > allocated to one person. You'll probably get a single digit number as > your answer. Can you calculate that using 32-bit integers? What about > 64-bit? Should you use bignums just in case? Take your pick, let me > know, and then I'll tell you my answer - and why. I'm not saying bignums are needed 0% of the time. But they don't need to be used for 100% of all integer values, just in case. > What sort of handicap is it? Performance? I don't know. Wasn't that brought up in the discussion? That Python's fib() routine could seamlessly accommodate bigger values as needed, and Julia's couldn't, and maybe that was one reason that Julia had the edge. Or maybe, in Bartville, this flexibility is > worth paying for but large integers are useless? In 40 years of coding I can say that I have very rarely needed arbitrary precision integer values. (They come up in some recreational programs, or for implementing arbitrary precision integers in the languages used to run those programs!) But I have needed flexible strings and lists all the time. Is my programming experience really that different from anybody else's? -- bartc From rosuav at gmail.com Fri Feb 23 08:34:06 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 00:34:06 +1100 Subject: atws In-Reply-To: References: <871shcfauk.fsf@handshake.de> Message-ID: On Sat, Feb 24, 2018 at 12:08 AM, Larry Martell wrote: > On Fri, Feb 23, 2018 at 2:01 AM, dieter wrote: >> Larry Martell writes: >>> ... >>> I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I get: >>> >>> No handlers could be found for logger "atws.connection" >> >> This is a warning (only), telling you that the "atws" package wants >> to log a message but there is not corresponding logging handler >> defined -- likely because you did not configure Python's logging >> infrastructure (--> "logging" package in Python's standard documentation). > > Yes, but this message comes from the import, which happens before any > of my code runs. It's perfectly legal to put other code before an import: import logging logging.basicConfig(...) import atws But this is just a warning, so it's not that big a deal either way. ChrisA From larry.martell at gmail.com Fri Feb 23 08:41:21 2018 From: larry.martell at gmail.com (Larry Martell) Date: Fri, 23 Feb 2018 08:41:21 -0500 Subject: atws In-Reply-To: References: <871shcfauk.fsf@handshake.de> Message-ID: On Fri, Feb 23, 2018 at 8:34 AM, Chris Angelico wrote: > On Sat, Feb 24, 2018 at 12:08 AM, Larry Martell wrote: >> On Fri, Feb 23, 2018 at 2:01 AM, dieter wrote: >>> Larry Martell writes: >>>> ... >>>> I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I get: >>>> >>>> No handlers could be found for logger "atws.connection" >>> >>> This is a warning (only), telling you that the "atws" package wants >>> to log a message but there is not corresponding logging handler >>> defined -- likely because you did not configure Python's logging >>> infrastructure (--> "logging" package in Python's standard documentation). >> >> Yes, but this message comes from the import, which happens before any >> of my code runs. > > It's perfectly legal to put other code before an import: > > import logging > logging.basicConfig(...) > import atws > > But this is just a warning, so it's not that big a deal either way. Yes, I know it's legal, but I've always thought it was bad form. From ben.usenet at bsb.me.uk Fri Feb 23 08:51:33 2018 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Fri, 23 Feb 2018 13:51:33 +0000 Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: <87bmgfyftm.fsf@bsb.me.uk> Steven D'Aprano writes: > On Fri, 23 Feb 2018 00:26:33 +0000, bartc wrote: > >> The point of the article was Julia vs. Python. You can't make Python >> faster by switching to a faster algorithm; you have to use the same one >> on both. > > No, the point of the article was to write Python code that is as fast as > the Julia code. I did not get that impression. In fact I was not really sure what the point was. At the start, the authors says "My take on this kind of cross language comparison is that the benchmarks should be defined by tasks to perform, then have language experts write the best code they can to perform these tasks." but if that is the point (rather the one on you picked up on) then the article could be, at best, only half the story. > I don't know why the Julia programmers chose to use such a poor > algorithm: It's odd indeed, but given that they did, what you take to be the point of the article -- to write a good Python algorithm as fast as the terrible Julia one -- seems a bit pointless. -- Ben. From python at bladeshadow.org Fri Feb 23 11:32:41 2018 From: python at bladeshadow.org (Python) Date: Fri, 23 Feb 2018 10:32:41 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> Message-ID: <20180223163241.GC10273@bladeshadow.org> On Fri, Feb 23, 2018 at 03:11:36AM -0500, Terry Reedy wrote: > >>Why do you care about the 50 million calls? That's crazy -- the important > >>thing is *calculating the Fibonacci numbers as efficiently as possible*. > > >If you are writing practical programs, that's true. But the Julia > >benchmarks are not practical programs; they are designed to compare > >the performance of various language features across a range of > >languages. > > If that were so, then the comparison should use the fastest *Python* > implementation. Doing that would completely fail to accomplish the task of comparing the performance of recursive function calls in the two languages, which is what the benchmark was designed to do. So, no actually, it shouldn't. From steve+comp.lang.python at pearwood.info Fri Feb 23 11:39:02 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 16:39:02 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Fri, 23 Feb 2018 23:41:44 +1100, Chris Angelico wrote: [...] >> Integer pixel values > > Maybe in 64 bits for the time being, but 32 certainly won't be enough. > As soon as you do any sort of high DPI image manipulation, you will > exceed 2**32 total pixels in an image (that's just 65536x65536, or > 46341x46341 if you're using signed integers); I don't think there's any sensible reason to use signed ints for pixel offsets. > and it wouldn't surprise > me if some image manipulation needs that many on a single side - if not > today, then tomorrow. So 64 bits might not be enough once you start > counting total pixels. 64-bit int will limit your image size to a maximum of 4294967296 x 4294967296. If you stitched the images from, let's say, the Hubble Space Telescope together into one panoramic image of the entire sky, you'd surely exceed that by a couple of orders of magnitude. There's a lot of sky and the resolution of the Hubble is very fine. Or made a collage of all the duck-face photos on Facebook *wink* But you wouldn't be able to open such a file on your computer. Not until we get a 128-bit OS :-) While all of this is very interesting, let's remember that listing all the many things which can be counted in 64 bits (or 128 bits, or 1024 bits...) doesn't disprove the need for bignums. The argument Bart makes is to list all the things that don't need refrigerating: - tinned beans - nuts and bolts - DVDs - books - spoons as evidence that we don't need refrigerators. Yeah, I get it, there are approximately a hundred thousand million billion trillion things that don't need bignums. There are so many things that don't need bignums, you need a bignum to count them all! I'm truly happy for Bart that he never, or almost never, uses numbers larger than 2**64. I just wish he would be less of a prig about this and stop trying to deny me my bignums. One of the things I like about Python is that I stopped needing to worry about integer overflow back around Python 2.2 when the default integer type became a bignum. I used to write code like: try: calculation(n) except OverflowError: calculation(long(n)) all the time back then. Now, if I have an integer calculation, I just calculate it without having to care whether it fits in 64 bits or not. And I don't give a rat's arse that this means that adding 1 to 10 to get 11 in Python takes thirty nanoseconds instead of ten nanoseconds as a consequence. If I cared about that, I wouldn't be using Python. Just the other day I need to calculate 23! (factorial) for a probability calculation, and Python made it trivially easy. I didn't need to import a special library, or use special syntax to say "use a bignum". I just multiplied 1 through 23. Why, just now, on a whim, simply because I can, I calculated 100!! (double-factorial): py> reduce(operator.mul, range(100, 0, -2), 1) 34243224702511976248246432895208185975118675053719198827915654463488000000000000 (Yes, I'm a maths geek, and I love this sort of thing. So sue me.) There are dozens of languages that have made the design choice to limit their default integers to 16- 32- or 64-bit fixed size, and let the user worry about overflow. Bart, why does it upset you so that Python made a different choice? -- Steve From rosuav at gmail.com Fri Feb 23 11:42:43 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 03:42:43 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <20180223163241.GC10273@bladeshadow.org> References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> Message-ID: On Sat, Feb 24, 2018 at 3:32 AM, Python wrote: > On Fri, Feb 23, 2018 at 03:11:36AM -0500, Terry Reedy wrote: >> >>Why do you care about the 50 million calls? That's crazy -- the important >> >>thing is *calculating the Fibonacci numbers as efficiently as possible*. >> >> >If you are writing practical programs, that's true. But the Julia >> >benchmarks are not practical programs; they are designed to compare >> >the performance of various language features across a range of >> >languages. >> >> If that were so, then the comparison should use the fastest *Python* >> implementation. > > Doing that would completely fail to accomplish the task of comparing > the performance of recursive function calls in the two languages, > which is what the benchmark was designed to do. So, no actually, it > shouldn't. > Where does the author say that the benchmark is designed to compare recursion? If you specifically test recursion, some language interpreters will perform really well (because they convert your recursion into iteration) and others will not (because they faithfully execute the code you actually ask them to). Recursion is sometimes a good way to describe an algorithm, but rarely a good way to implement it at a low level. ChrisA From python at bladeshadow.org Fri Feb 23 11:50:50 2018 From: python at bladeshadow.org (Python) Date: Fri, 23 Feb 2018 10:50:50 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223013621.GA10273@bladeshadow.org> Message-ID: <20180223165050.GD10273@bladeshadow.org> On Fri, Feb 23, 2018 at 01:38:01AM -0500, Terry Reedy wrote: > It is no secret that Python's interpreted function calls are slower > than function calls compiled to machine code and that Python's > infinite precision integer arithmetic is slower that machine int > arithmetic. So there is almost no point in combining both in a > naive drastically inefficient algorithm and declaring that Python is > slower. I never said the benchmarks chosen were awesome... :-) What I'm saying is this: 1. Insistence that the most efficient python implementation of Fib completely misses the point (and defeats the purpose) of the benchmarks, and as such the entire article is lost in the weeds. 2. Inasmuch as someone might want to measure the performance of recursive function calls in a particular language, the choice of recursively implementing fib is a fine one, and useful for that purpose. 3. If you want to say that choosing the most efficient implementation in Python is the only way to do a fair comparison, then you must also choose the most efficient implementation in Julia, which this is not, by all appearances. But again, this misses the point of the benchmark and defeats its purpose. Now if you want to complain that the purpose is silly, I won't even argue that... In 30 years of writing code I don't think I've ever imiplemented a recursive solution to any problem that wasn't purely academic. Maybe once or twice... maybe. But the purpose is what it is, and arguing that it must be done a different way is bogus. From rosuav at gmail.com Fri Feb 23 11:51:40 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 03:51:40 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On Sat, Feb 24, 2018 at 3:39 AM, Steven D'Aprano wrote: > On Fri, 23 Feb 2018 23:41:44 +1100, Chris Angelico wrote: > > [...] >>> Integer pixel values >> >> Maybe in 64 bits for the time being, but 32 certainly won't be enough. >> As soon as you do any sort of high DPI image manipulation, you will >> exceed 2**32 total pixels in an image (that's just 65536x65536, or >> 46341x46341 if you're using signed integers); > > I don't think there's any sensible reason to use signed ints for pixel > offsets. How about "my language's default integer type is signed"? That's what gave us all manner of problems, like the way a whole lot of old (and some not even all THAT old) programs have trouble if you have more than 2GB of free space - because they query disk free space as a 32-bit number, then interpret that number as signed. (If, as on many modern systems, you have hundreds or thousands of gig free, it depends whether that number modulo 4GB is more than 2GB, which is a very fun thing to manage.) >> and it wouldn't surprise >> me if some image manipulation needs that many on a single side - if not >> today, then tomorrow. So 64 bits might not be enough once you start >> counting total pixels. > > 64-bit int will limit your image size to a maximum of 4294967296 x > 4294967296. > > If you stitched the images from, let's say, the Hubble Space Telescope > together into one panoramic image of the entire sky, you'd surely exceed > that by a couple of orders of magnitude. There's a lot of sky and the > resolution of the Hubble is very fine. > > Or made a collage of all the duck-face photos on Facebook *wink* > > But you wouldn't be able to open such a file on your computer. Not until > we get a 128-bit OS :-) Hmm, yeah, I didn't actually do the math on that, heh. > While all of this is very interesting, let's remember that listing all > the many things which can be counted in 64 bits (or 128 bits, or 1024 > bits...) doesn't disprove the need for bignums. The argument Bart makes > is to list all the things that don't need refrigerating: > > - tinned beans > - nuts and bolts > - DVDs > - books > - spoons > > as evidence that we don't need refrigerators. Yeah, I get it, there are > approximately a hundred thousand million billion trillion things that > don't need bignums. There are so many things that don't need bignums, you > need a bignum to count them all! Touche! > And I don't give a rat's arse that this means that adding 1 to 10 to get > 11 in Python takes thirty nanoseconds instead of ten nanoseconds as a > consequence. If I cared about that, I wouldn't be using Python. Which is why you don't do crypto in Python - because you DO care about whether it takes thirty sometimes and ten sometimes, and various other things. So there *are* situations where you specifically do not want bignums; but they're a lot rarer than situations where you don't care what the underlying implementation is, and it's incredibly freeing to not have to take integer magnitude limitations into account. ChrisA From rosuav at gmail.com Fri Feb 23 11:55:06 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 03:55:06 +1100 Subject: atws In-Reply-To: References: <871shcfauk.fsf@handshake.de> Message-ID: On Sat, Feb 24, 2018 at 12:41 AM, Larry Martell wrote: > On Fri, Feb 23, 2018 at 8:34 AM, Chris Angelico wrote: >> On Sat, Feb 24, 2018 at 12:08 AM, Larry Martell wrote: >>> On Fri, Feb 23, 2018 at 2:01 AM, dieter wrote: >>>> Larry Martell writes: >>>>> ... >>>>> I had 2.2.1. I updated requests to 2.18.4 and now when I import atws I get: >>>>> >>>>> No handlers could be found for logger "atws.connection" >>>> >>>> This is a warning (only), telling you that the "atws" package wants >>>> to log a message but there is not corresponding logging handler >>>> defined -- likely because you did not configure Python's logging >>>> infrastructure (--> "logging" package in Python's standard documentation). >>> >>> Yes, but this message comes from the import, which happens before any >>> of my code runs. >> >> It's perfectly legal to put other code before an import: >> >> import logging >> logging.basicConfig(...) >> import atws >> >> But this is just a warning, so it's not that big a deal either way. > > Yes, I know it's legal, but I've always thought it was bad form. Most of the time, yes, you do all your imports at the top. But when you need to configure something before you import it, this is the best and simplest way to do it. Just try to keep it clean and simple, and you'll be fine. ChrisA From steve+comp.lang.python at pearwood.info Fri Feb 23 12:49:14 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 17:49:14 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <20180223033128.GB10273@bladeshadow.org> Message-ID: On Sat, 24 Feb 2018 00:03:06 +1100, Chris Angelico wrote: >> Is numpy a general purpose C library that can also be called from any >> language that can use a C API? Or is it specific to Python? >> >> > No, it's a general purpose FORTRAN library that can also be called from > any language that can use a generic C, FORTRAN, COBOL, etc API. Numpy itself is a collection of Python interfaces to a number of C and Fortran libraries. You can't generally call numpy from other languages -- you can't even call numpy from other Python implementations, unless they support calling C/Fortran. Jython, for example, can't do it without help: https://jyni.org/ and while PyPy can call numpy, doing so is slow, and there's a custom fork of numpy specially for PyPy: https://bitbucket.org/pypy/numpy Remember that numpy show cases the exact reason Python was invented: to act as an easy to use "glue language" to join together efficient and useful, but not so easy to use, libraries written in C and Fortran. Jython was invented to support Java libraries, and IronPython to make it easy to integrate with the Dot-Net ecosystem. -- Steve From steve+comp.lang.python at pearwood.info Fri Feb 23 13:00:33 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 18:00:33 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: Message-ID: On Fri, 23 Feb 2018 12:17:50 +0000, bartc wrote: > On 23/02/2018 01:27, Steven D'Aprano wrote: >> On Thu, 22 Feb 2018 17:53:30 +0000, bartc wrote: > >>> The actual result is irrelevant, so long as its correct. The important >>> thing is those 50 million calls. >> >> Why do you care about the 50 million calls? > > Because we are interested in comparing call-efficiency across languages? Are we? What on earth makes you think that? [...] > You're responsible for implementing the call-mechanism of a new > language, and you want to make it efficient. What program do you test it > with, one that contains zero calls (clearly, that will be most > efficient!), or one that contains lots of calls? This has nothing to do with the topic under consideration. It is irrelevant. This discussion has nothing, absolutely nothing with "implementing the call-mechanism of a new language". Perhaps you have missed that Python is a mature language well over 20 years old now, and even Julia is a few years old. [...] > OK, then, if you are at all interested in Julia vs. Python (and it was > you who posted that article!), then compare a version of a benchmark > that uses as few calls as possible, if you like. But what exactly is > that benchmark then highlighting? Loops? What on earth makes you think that the point of this is to highlight any one particular feature of a language? Loops, calls, addition, who cares? The point is to show how to take badly written slow code written in Python, and modify it to be better, faster code while remaining within the Python ecosystem. [...] > OK, then do as I did, and keep a global tally of how many calls it > makes, and print that as the result. Then the correct output for > 'fibcalls(36)' should be: > > 48315633 > > If yours shows only 37, then /your/ program is wrong. I'm not interested in such a function. The words have not yet been invented to explain how little I care about how fast Python or Julia can make millions and millions of pointless function calls for no good reason. But if it makes you feel good about yourself, if it makes you feel that you've won some sort of little victory, then I concede: If a programmer cares about writing useless, crappy, pointless code that just makes function call after function call for no good reason, then Python will be terrible for the job. I would never use Python for such code, it is awful at it. If you think it is important to run an empty loop a billion times in a nanosecond, Python will be crap at that too. Python is only useful for writing useful, meaningful code that gets useful work done, not for your "fibcalls" function. If you have a need for "fibcalls", then go ahead and use Julia, or C, or whatever language you prefer, and be glad that you have a wide choice of languages to choose from. -- Steve From steve+comp.lang.python at pearwood.info Fri Feb 23 13:05:59 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 18:05:59 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <87bmgfyftm.fsf@bsb.me.uk> Message-ID: On Fri, 23 Feb 2018 13:51:33 +0000, Ben Bacarisse wrote: [...] >> I don't know why the Julia programmers chose to use such a poor >> algorithm: > > It's odd indeed, but given that they did, what you take to be the point > of the article -- to write a good Python algorithm as fast as the > terrible Julia one -- seems a bit pointless. Someone looking at the Julia benchmarks will look at the results and go, "Wow! Julia is so much faster than Python, I should choose to use Julia instead!". But guess what? The benchmarks are flawed. The performance of real-world Julia code doesn't match the performance of the benchmarks. "What?s disappointing is the striking difference between the claimed performance and the observed one. For example, a trivial hello world program in Julia runs ~27x slower than Python?s version and ~187x slower than the one in C." http://zverovich.net/2016/05/13/giving-up-on-julia.html Admittedly "Hello World" is not what *I* would call a real world program, but that's just an illustration of the discrepancy between the performance in artificial benchmarks and the performance in useful code. The benchmarks emphasise what Julia is good at, and use poor, unidiomatic Python code. People who are either too naive to know better, or who *do* know better but for some bizarre reason still take benchmarks at face value, believe them, and conclude that Julia is faster than Python. Not just a bit faster, but enough to conclude that Python is uncompetitive. Okay, so Julia is fast, at least for toy benchmarks, maybe or maybe not for actual code you care about. Great. Python is fast too, if you stop writing shitty code. The *FIRST* lesson in optimization is to re-write your crap implementation with a better algorithm. The point of the post is to teach that lesson. Stop writing crap code and then complaining that the language is "too slow". Write better code, and then we'll take your complaints seriously. Or, you never know, maybe you'll find it's fast enough and you don't actually have to abandon your existing code base and migrate to a whole new language. "Python takes hours to sort a million integers using BubbleSort! Its too damn slow!!!" "Why not use the built-in sort? That's fast." "NOOOOOO!!!! I MUST USE BUBBLESORT, BECAUSE REASONS!!!1!" *wink* -- Steve From steve+comp.lang.python at pearwood.info Fri Feb 23 13:14:22 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 23 Feb 2018 18:14:22 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <20180223013621.GA10273@bladeshadow.org> <20180223165050.GD10273@bladeshadow.org> Message-ID: On Fri, 23 Feb 2018 10:50:50 -0600, Python wrote: > On Fri, Feb 23, 2018 at 01:38:01AM -0500, Terry Reedy wrote: >> It is no secret that Python's interpreted function calls are slower >> than function calls compiled to machine code and that Python's infinite >> precision integer arithmetic is slower that machine int arithmetic. >> So there is almost no point in combining both in a naive drastically >> inefficient algorithm and declaring that Python is slower. > > I never said the benchmarks chosen were awesome... :-) What I'm saying > is this: > > 1. Insistence that the most efficient python implementation of Fib > completely misses the point (and defeats the purpose) of the > benchmarks, and as such the entire article is lost in the weeds. The point of the benchmarks is to highlight how great Julia is, and why people should choose it over Python. The point of the article is to show how to take badly written, unidiomatic, slow code written in Python, and modify it to be better, faster code competitive (if not faster than) Julia, while remaining within the Python ecosystem. We aren't limited to the two options the benchmarks suggest: "Fast Julia! Slow Python!". (Three, if you include C.) We can take a third option: "we reject your crappy algorithms and write code like a professional". https://allthetropes.org/wiki/Take_a_Third_Option -- Steve From rosuav at gmail.com Fri Feb 23 13:20:34 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 05:20:34 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> Message-ID: On Sat, Feb 24, 2018 at 4:49 AM, Steven D'Aprano wrote: > On Sat, 24 Feb 2018 00:03:06 +1100, Chris Angelico wrote: > >>> Is numpy a general purpose C library that can also be called from any >>> language that can use a C API? Or is it specific to Python? >>> >>> >> No, it's a general purpose FORTRAN library that can also be called from >> any language that can use a generic C, FORTRAN, COBOL, etc API. > > Numpy itself is a collection of Python interfaces to a number of C and > Fortran libraries. You can't generally call numpy from other languages -- > you can't even call numpy from other Python implementations, unless they > support calling C/Fortran. Jython, for example, can't do it without help: > > https://jyni.org/ > > and while PyPy can call numpy, doing so is slow, and there's a custom > fork of numpy specially for PyPy: > > https://bitbucket.org/pypy/numpy Right. I was kinda handwaving a bit; numpy itself isn't the thing you'd call from elsewhere, but BLAS libraries in general are intended to be available from many languages. Point is, it's not Python-specific. ChrisA From rosuav at gmail.com Fri Feb 23 13:37:14 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 05:37:14 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> Message-ID: On Sat, Feb 24, 2018 at 5:05 AM, Steven D'Aprano wrote: > But guess what? The benchmarks are flawed. The performance of real-world > Julia code doesn't match the performance of the benchmarks. > > "What?s disappointing is the striking difference between > the claimed performance and the observed one. For example, > a trivial hello world program in Julia runs ~27x slower > than Python?s version and ~187x slower than the one in C." > > http://zverovich.net/2016/05/13/giving-up-on-julia.html > > Admittedly "Hello World" is not what *I* would call a real world program, > but that's just an illustration of the discrepancy between the > performance in artificial benchmarks and the performance in useful code. No, but Hello World is a good test of application startup time. (A null program doesn't always test everything, and doesn't look very good either.) If Hello World takes half a second to run, you have a language that is terrible for short scripts or command-line tools. That doesn't mean it's a "slow language" necessarily - nor even a slow interpreter, which is what you're actually testing - but it does mean that, well... rosuav at sikorsky:~$ time pypy -c 'print("Hello, world!")' Hello, world! real 0m0.429s user 0m0.036s sys 0m0.032s rosuav at sikorsky:~$ time python -c 'print("Hello, world!")' Hello, world! real 0m0.024s user 0m0.016s sys 0m0.008s ... CPython does have its place still. Or maybe... rosuav at sikorsky:~$ time pypy -c 'print("Hello, world!")' Hello, world! real 0m0.050s user 0m0.016s sys 0m0.032s rosuav at sikorsky:~$ time python -c 'print("Hello, world!")' Hello, world! real 0m0.022s user 0m0.016s sys 0m0.004s ... that PyPy doesn't live in my disk cache the way CPython does. :) PyPy is still slower than CPython for startup time, but not THAT much slower. However, if you DO have something that has a ton of startup overhead, you need to know about it. (Interestingly, I was actually able to compile and run a C hello-world in comparable time to PyPy, once the cache was warmed up. Curious. I expected that to be slower. Once again, it proves that intuition can be extremely wrong.) In terms of "which language is faster?", real-world code is both the ONLY way to measure, and a completely unfair comparison. Life is tough. ChrisA From python at bladeshadow.org Fri Feb 23 13:43:06 2018 From: python at bladeshadow.org (Python) Date: Fri, 23 Feb 2018 12:43:06 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> Message-ID: <20180223184306.GE10273@bladeshadow.org> On Sat, Feb 24, 2018 at 03:42:43AM +1100, Chris Angelico wrote: > >> If that were so, then the comparison should use the fastest *Python* > >> implementation. > > > > Doing that would completely fail to accomplish the task of comparing > > the performance of recursive function calls in the two languages, > > which is what the benchmark was designed to do. So, no actually, it > > shouldn't. > > > > Where does the author say that the benchmark is designed to compare > recursion? Chris, you're a smart guy... Are you suggesting that the reason the fibonacci sequence was selected as a benchmark is because it's such an amazingly useful problem that it, in and of itself, warrants having such a behchmark? Or, do you think the reason it makes sense to have such a benchmark is because, like the reason it's presented in pretty much every CS program ever, it presents an opportunity to consider a particular class of problems and different techniques for solving those problems, and the performance characteristics of those solutions? But, to answer your question more directly, here: https://julialang.org/benchmarks/ "It is important to note that the benchmark codes are not written for absolute maximal performance (the fastest code to compute recursion_fibonacci(20) is the constant literal 6765). Instead, the benchmarks are written to test the performance of identical algorithms and code patterns implemented in each language. For example, the Fibonacci benchmarks all use the same (inefficient) doubly-recursive algorithm..." Satisfied? > Recursion is sometimes a good way to describe an algorithm, but > rarely a good way to implement it at a low level. I'm well aware, and said the equivalent elsewhere. As I also said elsewhere, I never claimed it was a particularly useful benchmark. It is, nevertheless, designed to accomplish the stated goal, and does exactly that. You can decide for yourself how useful that goal is, but you can't really argue that it doesn't serve that purpose. So, by changing the algorithm, the article defeats the purpose of the benchmark. It makes some fine points about code optimization, but it completely fails at its stated purpose (to make the benchmarks more fair). The comparisons it makes are substantially less valid than the ones made by the Julia benchmarks, on account of optimizing only the algorithm used by Python, and not testing with a similarly optimized algorithm in Julia, but rather using its results for the intentionally unoptimized algorithm those benchmarks used. Even if testing optimized code is the point, as the article claims, it utterly fails to do that. Bad science. From rosuav at gmail.com Fri Feb 23 13:56:25 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 05:56:25 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <20180223184306.GE10273@bladeshadow.org> References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> Message-ID: On Sat, Feb 24, 2018 at 5:43 AM, Python wrote: > On Sat, Feb 24, 2018 at 03:42:43AM +1100, Chris Angelico wrote: >> >> If that were so, then the comparison should use the fastest *Python* >> >> implementation. >> > >> > Doing that would completely fail to accomplish the task of comparing >> > the performance of recursive function calls in the two languages, >> > which is what the benchmark was designed to do. So, no actually, it >> > shouldn't. >> > >> >> Where does the author say that the benchmark is designed to compare >> recursion? > > Chris, you're a smart guy... Are you suggesting that the reason the > fibonacci sequence was selected as a benchmark is because it's such an > amazingly useful problem that it, in and of itself, warrants having > such a behchmark? Or, do you think the reason it makes sense to have > such a benchmark is because, like the reason it's presented in pretty > much every CS program ever, it presents an opportunity to consider a > particular class of problems and different techniques for solving > those problems, and the performance characteristics of those > solutions? > > > But, to answer your question more directly, here: > > https://julialang.org/benchmarks/ > > "It is important to note that the benchmark codes are not written > for absolute maximal performance (the fastest code to compute > recursion_fibonacci(20) is the constant literal 6765). Instead, > the benchmarks are written to test the performance of identical > algorithms and code patterns implemented in each language. For > example, the Fibonacci benchmarks all use the same (inefficient) > doubly-recursive algorithm..." > > Satisfied? No, not satisfied. Everything you've said would still be satisfied if all versions of the benchmark used the same non-recursive algorithm. There's nothing here that says it's testing recursion, just that (for consistency) it's testing the same algorithm. There is no reason to specifically test *recursion*, unless that actually aligns with what you're doing. For instance, when Google Chrome rolled out its new V8 JavaScript engine, it was specifically optimized for recursion, because many Google apps used recursion heavily. If you're implementing a new LISP interpreter, you should probably optimize for recursion, because most LISP code is going to be written recursively. But otherwise, there's no particular reason to stick to recursion. > So, by changing the algorithm, the article defeats the purpose of the > benchmark. It makes some fine points about code optimization, but it > completely fails at its stated purpose (to make the benchmarks more > fair). The comparisons it makes are substantially less valid than the > ones made by the Julia benchmarks, on account of optimizing only the > algorithm used by Python, and not testing with a similarly optimized > algorithm in Julia, but rather using its results for the intentionally > unoptimized algorithm those benchmarks used. Even if testing > optimized code is the point, as the article claims, it utterly fails > to do that. Bad science. > I won't dispute that part. The correct way to do this would be to optimize both sides fairly - either not at all, or equivalently (for some definition of equivalence). But switching both sides to an unoptimized iterative algorithm would be perfectly fair. Recursion is NOT essential to the benchmark. ChrisA From python at bladeshadow.org Fri Feb 23 14:03:34 2018 From: python at bladeshadow.org (Python) Date: Fri, 23 Feb 2018 13:03:34 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223013621.GA10273@bladeshadow.org> <20180223165050.GD10273@bladeshadow.org> Message-ID: <20180223190334.GF10273@bladeshadow.org> On Fri, Feb 23, 2018 at 06:14:22PM +0000, Steven D'Aprano wrote: > > I never said the benchmarks chosen were awesome... :-) What I'm saying > > is this: > > > > 1. Insistence that the most efficient python implementation of Fib > > completely misses the point (and defeats the purpose) of the > > benchmarks, and as such the entire article is lost in the weeds. > > The point of the benchmarks is to highlight how great Julia is For a particular set of common code patterns. You left that part out, and it's the crux of the issue. > and why people should choose it over Python. I'd argue not why, but WHEN. Or really I'd argue nothing of the sort... and simply point out that the benchmarks do exactly what the Julia team say they do. It's up to you to decide how useful that is. > The point of the article is to show how to take badly written, > unidiomatic, slow code written in Python, and modify it to be better, > faster code competitive (if not faster than) Julia, while remaining > within the Python ecosystem. Even if that was the point, the article fails, because it compares non-optimized Julia code to optimized Pyhton code. > We aren't limited to the two options the benchmarks suggest It actually suggested quite a lot more than that... benchmarks compared the performance of 12 languages, using exactly the same programming patterns (as near as I can tell anyway, I didn't scrutinize all of the code). From python at bladeshadow.org Fri Feb 23 14:09:16 2018 From: python at bladeshadow.org (Python) Date: Fri, 23 Feb 2018 13:09:16 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> Message-ID: <20180223190916.GG10273@bladeshadow.org> On Sat, Feb 24, 2018 at 05:56:25AM +1100, Chris Angelico wrote: > No, not satisfied. Everything you've said would still be satisfied if > all versions of the benchmark used the same non-recursive algorithm. > There's nothing here that says it's testing recursion, just that (for > consistency) it's testing the same algorithm. There is no reason to > specifically test *recursion*, unless that actually aligns with what > you're doing. It seems abundantly clear to me that testing recursion is the point of writing a benchmark implementing recursion (and very little of anything else). Again, you can decide for yourself the suitability of the benchmark, but I don't think you can really claim it doesn't effectively test what it means to. From rosuav at gmail.com Fri Feb 23 14:16:38 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 06:16:38 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <20180223190916.GG10273@bladeshadow.org> References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <20180223190916.GG10273@bladeshadow.org> Message-ID: On Sat, Feb 24, 2018 at 6:09 AM, Python wrote: > On Sat, Feb 24, 2018 at 05:56:25AM +1100, Chris Angelico wrote: >> No, not satisfied. Everything you've said would still be satisfied if >> all versions of the benchmark used the same non-recursive algorithm. >> There's nothing here that says it's testing recursion, just that (for >> consistency) it's testing the same algorithm. There is no reason to >> specifically test *recursion*, unless that actually aligns with what >> you're doing. > > It seems abundantly clear to me that testing recursion is the point of > writing a benchmark implementing recursion (and very little of > anything else). Again, you can decide for yourself the suitability of > the benchmark, but I don't think you can really claim it doesn't > effectively test what it means to. Where do you get that it's specifically a recursion benchmark though? I can't find it anywhere in the explanatory text. ChrisA From bc at freeuk.com Fri Feb 23 14:25:35 2018 From: bc at freeuk.com (bartc) Date: Fri, 23 Feb 2018 19:25:35 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> Message-ID: On 23/02/2018 18:05, Steven D'Aprano wrote: > On Fri, 23 Feb 2018 13:51:33 +0000, Ben Bacarisse wrote: > Stop writing crap code and then complaining that the language is "too > slow". Write better code, and then we'll take your complaints seriously. I write compilers for static languages, which are generally considered toys, partly because the code generated is not great. But in a real application, the difference between my compiler, and gcc-O3 or MSVC/O2 might only be 10-50%. 1.1 to 1.5 times as fast (or as slow), and that's an application that does few external library calls. The difference between Python and another dynamic language might be a magnitude, yet you say it doesn't matter. Thanks, that makes me feel much better about my own work! If a magnitude difference in performance can be so casually dismissed. > "Python takes hours to sort a million integers using BubbleSort! Its too > damn slow!!!" > > "Why not use the built-in sort? That's fast." > > "NOOOOOO!!!! I MUST USE BUBBLESORT, BECAUSE REASONS!!!1!" > > *wink* Have you ever written a benchmark in your life? If so, did it do anything useful? Other than what benchmarks are normally used for. -- bartc From bc at freeuk.com Fri Feb 23 14:36:39 2018 From: bc at freeuk.com (bartc) Date: Fri, 23 Feb 2018 19:36:39 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> Message-ID: On 23/02/2018 18:56, Chris Angelico wrote: > On Sat, Feb 24, 2018 at 5:43 AM, Python wrote: >> Satisfied? > > No, not satisfied. Everything you've said would still be satisfied if > all versions of the benchmark used the same non-recursive algorithm. Why? Does it matter? > There's nothing here that says it's testing recursion, just that (for > consistency) it's testing the same algorithm. There is no reason to > specifically test *recursion*, unless that actually aligns with what > you're doing. For instance, when Google Chrome rolled out its new V8 > JavaScript engine, it was specifically optimized for recursion, > because many Google apps used recursion heavily. If you're > implementing a new LISP interpreter, you should probably optimize for > recursion, because most LISP code is going to be written recursively. > But otherwise, there's no particular reason to stick to recursion. You list plenty of reasons, then say there's no reason to use recursion! The recursion is a red herring; but to get a program with lots of function calling, then using a recursive algorithm - any algorithm - will do the job. Would you have a different opinion if Python excelled at that benchmark, and Julia sucked? (I can't remember what the results were.) (I was testing one of my static compilers today; it has three call convention modes all operating within the Win64 ABI. I used the Fibonacci benchmark with fib(42) to test it, with these results: mode 1 5.8 seconds (non-ABI) mode 2 6.5 seconds (part ABI conformance) mode 3 7.4 seconds (full ABI conformance) Other compilers (C compilers that have to conform) ranged from 4.3 to 6.2 seconds, including MSVC/O2. Except for gcc-O3 which managed 1.7 seconds. How do it do that? I'll have to investigate to see how much it cheated. The point of all this: I was using the recursive Fibonacci benchmark for these tests, as there is little else going on besides function calls and returns. It's perfect. Why some people hate it here, I really don't know.) > I won't dispute that part. The correct way to do this would be to > optimize both sides fairly - either not at all, or equivalently (for > some definition of equivalence). But switching both sides to an > unoptimized iterative algorithm would be perfectly fair. Recursion is > NOT essential to the benchmark. So, what benchmark would you use instead if you wanted an idea of how well each language dealt with it? -- bartc From python at bladeshadow.org Fri Feb 23 14:41:17 2018 From: python at bladeshadow.org (Python) Date: Fri, 23 Feb 2018 13:41:17 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <20180223190916.GG10273@bladeshadow.org> Message-ID: <20180223194117.GH10273@bladeshadow.org> On Sat, Feb 24, 2018 at 06:16:38AM +1100, Chris Angelico wrote: > > It seems abundantly clear to me that testing recursion is the point of > > writing a benchmark implementing recursion (and very little of > > anything else). Again, you can decide for yourself the suitability of > > the benchmark, but I don't think you can really claim it doesn't > > effectively test what it means to. > > Where do you get that it's specifically a recursion benchmark though? > I can't find it anywhere in the explanatory text. I don't know how you could conclude it could be meant to test anything else. The code is: ## recursive fib ## fib(n) = n < 2 ? n : fib(n-1) + fib(n-2) What else is there of substance to test? Trigraphs? Addition? The comment even calls out that it is recursive. I further draw inference from the two statements: 1. "Instead, the benchmarks are written to test the performance of identical algorithms and code patterns implemented in each language." Calling out explicitly that it is algorithms that they are testing, and here the algorithm is a recursion. 2. "For example, the Fibonacci benchmarks all use the same (inefficient) doubly-recursive algorithm..." They expressly called out this algorithm as being inefficient, and expressly called it out as being double-recursion. I'll grant you they never explicitly said so unequivocally (that I saw--my reading of their site was not exhaustive), but I can't see any way an informed reader could conclude any different purpose other than benchmarking recursion. From rosuav at gmail.com Fri Feb 23 14:47:00 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 06:47:00 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> Message-ID: On Sat, Feb 24, 2018 at 6:25 AM, bartc wrote: > On 23/02/2018 18:05, Steven D'Aprano wrote: >> >> On Fri, 23 Feb 2018 13:51:33 +0000, Ben Bacarisse wrote: > > >> Stop writing crap code and then complaining that the language is "too >> slow". Write better code, and then we'll take your complaints seriously. > > > I write compilers for static languages, which are generally considered toys, > partly because the code generated is not great. Doesn't matter what language you're writing in, if your algorithm is flawed. Bubble sort is bubble sort no matter what you're using. This function will calculate triangle numbers just as inefficiently in Python as it would in C: def triangle(n): if n <= 1: return n return triangle(n-1) + triangle(n-1) - triangle(n-2) + 1 > But in a real application, the difference between my compiler, and gcc-O3 or > MSVC/O2 might only be 10-50%. 1.1 to 1.5 times as fast (or as slow), and > that's an application that does few external library calls. > > The difference between Python and another dynamic language might be a > magnitude, yet you say it doesn't matter. > > Thanks, that makes me feel much better about my own work! If a magnitude > difference in performance can be so casually dismissed. Yep! Your work is awesome, because your code is only a little bit slower than the same algorithm with the same bugs in a better-known language that's supported on more platforms and has a large team of people supporting it. Remind me what's actually advantageous about your language? ChrisA From jack at alcor.concordia.ca Fri Feb 23 14:58:25 2018 From: jack at alcor.concordia.ca (Jack Fearnley) Date: Fri, 23 Feb 2018 19:58:25 -0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <3cf440d6-ca55-40c1-9bc7-a58ade413bbb@googlegroups.com> Message-ID: On Thu, 22 Feb 2018 17:13:02 -0800, Rick Johnson wrote: > On Thursday, February 22, 2018 at 1:55:35 PM UTC-6, Jack Fearnley wrote: > [...] >> I realize that this thread is about benchmarking and not really about >> generating fibonacci numbers, but I hope nobody is using this code to >> generate them on a 'production' basis, > > I've been raising the warning flags about that awful Fibonacci example > for years! > > First it was in the tutorial, and then, they plastered it on the front > page of Python.org like it were some sort of trophy... > > At this point i have concluded that it must be: (1) a sadistic noob > hazing device, or (2) a straw-stuffed, > anthropomorphic, crow-poop-riddled monster that sends the seasoned > programmers off in the opposite direction. Excellent thread! I see I'm preaching to the converted. Jack From python at bladeshadow.org Fri Feb 23 15:02:09 2018 From: python at bladeshadow.org (Python) Date: Fri, 23 Feb 2018 14:02:09 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <20180223194117.GH10273@bladeshadow.org> References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <20180223190916.GG10273@bladeshadow.org> <20180223194117.GH10273@bladeshadow.org> Message-ID: <20180223200209.GI10273@bladeshadow.org> On Fri, Feb 23, 2018 at 01:41:17PM -0600, Python wrote: > On Sat, Feb 24, 2018 at 06:16:38AM +1100, Chris Angelico wrote: > > > It seems abundantly clear to me that testing recursion is the point of > > > writing a benchmark implementing recursion (and very little of > > > anything else). Again, you can decide for yourself the suitability of > > > the benchmark, but I don't think you can really claim it doesn't > > > effectively test what it means to. > > > > Where do you get that it's specifically a recursion benchmark though? > > I can't find it anywhere in the explanatory text. > > I don't know how you could conclude it could be meant to test anything > else. Actually I will make one small correction, which is to say that the benchmark tests the performance of algorithms which are predominated by many function calls, of which recursion is an obvious subset. But I think that's splitting hairs. From bc at freeuk.com Fri Feb 23 15:02:21 2018 From: bc at freeuk.com (bartc) Date: Fri, 23 Feb 2018 20:02:21 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> Message-ID: <1r_jC.206183$HI2.106302@fx07.am4> On 23/02/2018 19:47, Chris Angelico wrote: > On Sat, Feb 24, 2018 at 6:25 AM, bartc wrote: >> The difference between Python and another dynamic language might be a >> magnitude, yet you say it doesn't matter. >> >> Thanks, that makes me feel much better about my own work! If a magnitude >> difference in performance can be so casually dismissed. > > Yep! Your work is awesome, because your code is only a little bit > slower than the same algorithm with the same bugs in a better-known > language that's supported on more platforms and has a large team of > people supporting it. > > Remind me what's actually advantageous about your language? I don't know what point you're making here. Unless it's that no software is worth writing unless it's done on a big scale with a huge team of people, and is very well known. My own point should be clear: Python is 10 times slower than a competitor = doesn't matter My language is 1.5 times slower than the big boys' = matters a great deal From rosuav at gmail.com Fri Feb 23 15:12:48 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 07:12:48 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <1r_jC.206183$HI2.106302@fx07.am4> References: <87bmgfyftm.fsf@bsb.me.uk> <1r_jC.206183$HI2.106302@fx07.am4> Message-ID: On Sat, Feb 24, 2018 at 7:02 AM, bartc wrote: > On 23/02/2018 19:47, Chris Angelico wrote: >> >> On Sat, Feb 24, 2018 at 6:25 AM, bartc wrote: > > >>> The difference between Python and another dynamic language might be a >>> magnitude, yet you say it doesn't matter. >>> >>> Thanks, that makes me feel much better about my own work! If a magnitude >>> difference in performance can be so casually dismissed. >> >> >> Yep! Your work is awesome, because your code is only a little bit >> slower than the same algorithm with the same bugs in a better-known >> language that's supported on more platforms and has a large team of >> people supporting it. >> >> Remind me what's actually advantageous about your language? > > > I don't know what point you're making here. Unless it's that no software is > worth writing unless it's done on a big scale with a huge team of people, > and is very well known. > > My own point should be clear: > > Python is 10 times slower than a competitor = doesn't matter > My language is 1.5 times slower than the big boys' = matters a great deal Most performance is a tradeoff. Python has other advantages; what are yours? So far, all you've said is that you're "not too much worse". That's great for a toy language! ChrisA From ned at nedbatchelder.com Fri Feb 23 15:40:06 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Fri, 23 Feb 2018 15:40:06 -0500 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <1r_jC.206183$HI2.106302@fx07.am4> References: <87bmgfyftm.fsf@bsb.me.uk> <1r_jC.206183$HI2.106302@fx07.am4> Message-ID: <8cc9a1cc-922a-6efd-e4dd-b5e4b530aa7f@nedbatchelder.com> On 2/23/18 3:02 PM, bartc wrote: > On 23/02/2018 19:47, Chris Angelico wrote: >> On Sat, Feb 24, 2018 at 6:25 AM, bartc wrote: > >>> The difference between Python and another dynamic language might be a >>> magnitude, yet you say it doesn't matter. >>> >>> Thanks, that makes me feel much better about my own work! If a >>> magnitude >>> difference in performance can be so casually dismissed. >> >> Yep! Your work is awesome, because your code is only a little bit >> slower than the same algorithm with the same bugs in a better-known >> language that's supported on more platforms and has a large team of >> people supporting it. >> >> Remind me what's actually advantageous about your language? > > I don't know what point you're making here. Unless it's that no > software is worth writing unless it's done on a big scale with a huge > team of people, and is very well known. > > My own point should be clear: > > ?Python is 10 times slower than a competitor = doesn't matter > ?My language is 1.5 times slower than the big boys' = matters a great > deal > > I've lost track of the points anyone here is making.? Can we PLEASE avoid getting into this tarpit of comparing Bart's language to Python yet again? --Ned. From bc at freeuk.com Fri Feb 23 16:32:50 2018 From: bc at freeuk.com (bartc) Date: Fri, 23 Feb 2018 21:32:50 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> <1r_jC.206183$HI2.106302@fx07.am4> Message-ID: On 23/02/2018 20:12, Chris Angelico wrote: > On Sat, Feb 24, 2018 at 7:02 AM, bartc wrote: >> I don't know what point you're making here. Unless it's that no software is >> worth writing unless it's done on a big scale with a huge team of people, >> and is very well known. >> >> My own point should be clear: >> >> Python is 10 times slower than a competitor = doesn't matter >> My language is 1.5 times slower than the big boys' = matters a great deal > > Most performance is a tradeoff. Python has other advantages; what are > yours? So far, all you've said is that you're "not too much worse". > That's great for a toy language! Ned Batchelder has banned me from mentioning any of my private work here. So I'll keep it generic. Let's say the Tiny C compiler is not taken seriously because it might be a couple of times slower than gcc-O3, even thought it's 1% of the size and compiles 1000% as fast. But the difference in runtime speed between Python and other dynamic languages, if you look at benchmarks doing actual work in the language, can be much greater than two times, yet that doesn't appear to matter. Even defining how fast or slow Python actually is is fraught with complexities making it hard to establish just what its comparative speed is: No one seems to be able to agree with what benchmarks ought to be used. Some don't seem to get benchmarks. Then it's question of whether you run CPython. Or PyPy. Or Ironpython or Jpython. Or Numba with @jit. Or Numpy. Or @cache or whatever it is. Or you run Cython. Or maybe you can combine @jit, @cache, Cython, and PyPy in the same program; I don't know. (Have I ever mentioned Python is complicated?) So, you were asking about the benefits of having a small, simple self-contained implementation of a language. I really couldn't tell you... -- bartc From rosuav at gmail.com Fri Feb 23 16:51:27 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Feb 2018 08:51:27 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> <1r_jC.206183$HI2.106302@fx07.am4> Message-ID: On Sat, Feb 24, 2018 at 8:32 AM, bartc wrote: > So I'll keep it generic. Let's say the Tiny C compiler is not taken > seriously because it might be a couple of times slower than gcc-O3, even > thought it's 1% of the size and compiles 1000% as fast. Except that nobody has said that. You're doing an excellent job of fighting against a strawman. I'm done. Have fun! ChrisA From drsalists at gmail.com Fri Feb 23 19:45:38 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 23 Feb 2018 16:45:38 -0800 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> <1r_jC.206183$HI2.106302@fx07.am4> Message-ID: On Fri, Feb 23, 2018 at 1:32 PM, bartc wrote: > On 23/02/2018 20:12, Chris Angelico wrote: > So I'll keep it generic. Let's say the Tiny C compiler is not taken > seriously because it might be a couple of times slower than gcc-O3, even > thought it's 1% of the size and compiles 1000% as fast. What's generic about a specific example? (that's rhetorical) I've used tcc in preference to gcc before; I had an application where performance didn't matter as much as security; tcc has buffer overrun checking. > But the difference in runtime speed between Python and other dynamic > languages, if you look at benchmarks doing actual work in the language, can > be much greater than two times, yet that doesn't appear to matter. The best way to use Python is to: 1) Write your whole application in it. 2) IF things are too slow, profile the app. Usually performance will be fast enough from the start, but not always. 3) After (if) identifying a hotspot, optimize it in any of a variety of ways, while still reaping the benefits of rapid development (which often matters much more than CPU speed today) I have a page about speeding up python: http://stromberg.dnsalias.org/~strombrg/speeding-python/ But again, those techniques are only infrequently relevant, and pretty much never relevant to an entire application. From steve+comp.lang.python at pearwood.info Fri Feb 23 21:05:17 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 24 Feb 2018 02:05:17 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <87bmgfyftm.fsf@bsb.me.uk> Message-ID: On Fri, 23 Feb 2018 19:25:35 +0000, bartc wrote: > On 23/02/2018 18:05, Steven D'Aprano wrote: >> On Fri, 23 Feb 2018 13:51:33 +0000, Ben Bacarisse wrote: > >> Stop writing crap code and then complaining that the language is "too >> slow". Write better code, and then we'll take your complaints >> seriously. > > I write compilers for static languages, which are generally considered > toys, partly because the code generated is not great. I have no idea whether your compilers are toys, or what "not great" means for your generated code. - too slow for the intended purpose? - full of bugs? - generated code is enormous? - requires too much memory? - all of the above? If the actual answer is, "none of the above", then apart from your failure to attract users and build a full ecosystem around your compiler, I see no reason to call them toys. If your compilers can get within 10% of best-of-breed C compilers, even if only on trivial problems, that's nothing to sneer at. If you can do it on non-trivial applications (say, the Linux kernel) without introducing a vast number of compiler-generated bugs, then you're doing very well indeed. In another post (responding to Chris) you wrote: > Python is 10 times slower than a competitor = doesn't matter > My language is 1.5 times slower than the big boys' = matters > a great deal Maybe it matters to *you*. You are the one who stated it was a toy compiler. To me, I have *no idea* whether the compiler is any good. Nobody but you uses it. I once tried to compile it, and couldn't get it to compile. As for Python's order-of-magnitude speed difference, thank you for being generous. If you cherry-pick the right benchmarks, you could easily have said it was two orders-of-magnitude slower. On the other hand, if you use PyPy, and similarly cherry-pick the right benchmarks, you can justifiably say that *Python is faster than C*. (On carefully chosen examples which probably aren't representative of full application performance.) But okay, let's take "Python is an order of magnitude (between 10 and 90 times) slower than C" for the sake of the argument. Does it matter? Maybe. Maybe not. Sometimes it matters. Sometimes it doesn't. The world's fastest street-legal production car is, so I'm told, the Bugatti Veyron Super Sport. It has a top speed of 268 mph and the engine has 1200 horsepower. And yet literally *billions* of car drivers choose to drive something slower. They wouldn't buy a Veyron even if they could afford it. Why shouldn't people make choices based on factors other than speed, speed, speed, speed, speed? Ruby has fallen out of favour now, but for quite a few years it was looking like a contender despite being an order of magnitude slower than Python. Which is an order of magnitude slower than Java, which is an order of magnitude slower than C, according to some benchmarks at least. People write business applications using Excel spreadsheets and macros. And why not? So it is quite possible to get practical work done and be a competitive, useful language despite being (allegedly) a thousand or more times slower than C. [...] > Have you ever written a benchmark in your life? If so, did it do > anything useful? Other than what benchmarks are normally used for. Yes I have, not for what benchmarks are normally used for (dick-measuring contests between programmers competing to see whose language is more macho) but for the very practical purposes of: (1) Deciding which algorithm to use in practice; (2) Ensuring that updates to the code don't suffer serious performance regressions; (3) And checking that the code is *fast enough* (not as fast as conceivably possible) for its purpose. They are the sorts of benchmarks I care about. -- Steve From bc at freeuk.com Fri Feb 23 21:06:05 2018 From: bc at freeuk.com (bartc) Date: Sat, 24 Feb 2018 02:06:05 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> <1r_jC.206183$HI2.106302@fx07.am4> Message-ID: <0M3kC.199533$C92.53838@fx37.am4> On 24/02/2018 00:45, Dan Stromberg wrote: > On Fri, Feb 23, 2018 at 1:32 PM, bartc wrote: >> But the difference in runtime speed between Python and other dynamic >> languages, if you look at benchmarks doing actual work in the language, can >> be much greater than two times, yet that doesn't appear to matter. > > The best way to use Python is to: > 1) Write your whole application in it. > 2) IF things are too slow, profile the app. Usually performance will > be fast enough from the start, but not always. > 3) After (if) identifying a hotspot, optimize it in any of a variety > of ways, while still reaping the benefits of rapid development (which > often matters much more than CPU speed today) > > I have a page about speeding up python: > http://stromberg.dnsalias.org/~strombrg/speeding-python/ > > But again, those techniques are only infrequently relevant, and pretty > much never relevant to an entire application. That your page lists 23 different approaches to making Python faster suggests that its speed can be an issue. Certainly there seem to be a lot of projects going on trying to fix that aspect. So I'll rephrase, and say that some individuals in this group are saying that speed doesn't matter. For many others it apparently does. (I've just looked at the benchmark results referred to in the OP's link [https://julialang.org/benchmarks/] and the comparison for the recursion benchmark (let's just call it that) seems fair enough to me. Perhaps it ought to have been clear that this was CPython, if it in fact was. There are so many possibilities with Python, that using anything else would confuse matters. People who use Python will be aware there are acceleration techniques. (A note regarding the C timing in that table: I've looked at what gcc does with this benchmark with its unreasonably fast timing, and it appears to cheat. That is, it does not call that function (ie. enter via the normal prologue code) the number of times it says. I don't know if that was the case when they tested C for that table. But it can happen that a sufficiently simple benchmark can lend itself to super-optimisation that is much harder with real code. For that reason I no longer take account of gcc-O3 timings for micro-benchmarks.)) -- bartc From drsalists at gmail.com Fri Feb 23 21:22:29 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 23 Feb 2018 18:22:29 -0800 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <0M3kC.199533$C92.53838@fx37.am4> References: <87bmgfyftm.fsf@bsb.me.uk> <1r_jC.206183$HI2.106302@fx07.am4> <0M3kC.199533$C92.53838@fx37.am4> Message-ID: On Fri, Feb 23, 2018 at 6:06 PM, bartc wrote: > On 24/02/2018 00:45, Dan Stromberg wrote: >> But again, those techniques are only infrequently relevant, and pretty >> much never relevant to an entire application. > That your page lists 23 different approaches to making Python faster > suggests that its speed can be an issue. Certainly there seem to be a lot of > projects going on trying to fix that aspect. Of course CPU performance can be an issue. For any language. When I was a kid, programs were written in BASIC. If BASIC was too slow, you'd rewrite the hotspots in assembler. Does that mean everything should have been written in assembler? Not at all. Assembler was slow to code in and tended to produce crashy solutions. Obsessing about CPU time is outdated thinking for almost all problems. Developer time is much more likely to matter to the bottom line > Perhaps it ought to have been clear that this was CPython, if it in fact > was. CPython != Python. g++ != C++. gcc != C. Python is a language. Not an implementation of a language. > There are so many possibilities with Python, that using anything else > would confuse matters. Gosh no. A language's success can, in significant part, be judged by the number of compatible implementations and dialects. You might find this interesting: http://stromberg.dnsalias.org/~strombrg/why-is-python-slow/ It's just one microbenchmark, but it's me addressing someone asking why Python is so much slower than C++. From steve+comp.lang.python at pearwood.info Fri Feb 23 21:46:02 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 24 Feb 2018 02:46:02 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> Message-ID: On Fri, 23 Feb 2018 12:43:06 -0600, Python wrote: > Even if testing optimized > code is the point, as the article claims, it utterly fails to do that. > Bad science. You've used that statement two or three times now. *This isn't science*. There's nothing scientific about writing benchmarks, or even objective. It is through and through subjective choices given a paper-thin patina of objectivity because the results include numbers. But those numbers depend on the precise implementation of the benchmark. They depend on the machine you run them on, sometimes strongly enough that the order of which language is faster can swap. I remember a bug in Python's urllib module, I think it was, that made code using it literally hundreds of times slower on Windows than Linux or OS X. The choice of algorithms used is not objective, or fair. Most of it is tradition: the famous "whetstone" benchmark apparently measures something which has little or no connection to anything software developers should care about. It, like the Dhrystone variant, were invented to benchmark CPU performance. The relevance to comparing languages is virtually zero. "As this data reveals, Dhrystone is not a particularly representative sample of the kinds of instruction sequences that are typical of today's applications. The majority of embedded applications make little use of the C libraries for example, and even desktop applications are unlikely to have such a high weighting of a very small number of specific library calls." http://dell.docjava.com/courses/cr346/data/papers/DhrystoneMIPS- CriticismbyARM.pdf Take the Fibonacci double-recursion benchmark. Okay, it tests how well your language does at making millions of function calls. Why? How often do you make millions of function calls? For most application code, executing the function is far more costly than the overhead of calling it, and the call overhead is dwarfed by the rest of the application. For many, many applications, the *entire* program run could take orders of magnitude fewer function calls than a single call to fib(38). If you have a language with tail recursion elimination, you can bet that's its benchmarks will include examples of tail recursion and tail recursion will be a favoured idiom in that language. If it doesn't, it won't. I'm going to end with a quote: "And of course, the very success of a benchmark program is a danger in that people may tune their compilers and/or hardware to it, and with this action make it less useful." Reinhold P. Weicker, Siemens AG, April 1989 Author of the Dhrystone Benchmark -- Steve From tjreedy at udel.edu Sat Feb 24 00:18:58 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 24 Feb 2018 00:18:58 -0500 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <20180223163241.GC10273@bladeshadow.org> References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> Message-ID: On 2/23/2018 11:32 AM, Python wrote: > On Fri, Feb 23, 2018 at 03:11:36AM -0500, Terry Reedy wrote: >>>> Why do you care about the 50 million calls? That's crazy -- the important >>>> thing is *calculating the Fibonacci numbers as efficiently as possible*. >> >>> If you are writing practical programs, that's true. But the Julia >>> benchmarks are not practical programs; they are designed to compare >>> the performance of various language features across a range of >>> languages. >> >> If that were so, then the comparison should use the fastest *Python* >> implementation. > > Doing that would completely fail to accomplish the task of comparing > the performance of recursive function calls in the two languages, > which is what the benchmark was designed to do. That is an non goal because *languages* do not have clock-time speeds, only programs written in concrete implementations run on real hardware. Why do you think it fair to compare function call times using the slowest rather than the fastest implementation of Python function calls? Real Python programmers who are seriously concerned about time try to use the fastest implementation appropriate to the situation. > So, no actually, it shouldn't. To me, it is 'not using the fasted Python' that fails to make apples to apples comparisons. It has been said here that comparisons should use the same algorithm doing the much the same work in both implementation. However, a Python function call does *much more work* than in most languages, because the meaning of 'function call' is much more flexible in Python than most languages. The usual comparison is like lemons (other language calls) to oranges (Python language calls, much more flexible). To be fair, the comparison should be to a Python implementation that either notices or accepts a declaration that, for instance, fib(n) only needs to pass an int by position. Comparing int addition to python object addition also compares unlike operations. To compare using the same addition algorithm, one must use an implementation that can restrict '+' to just int addition. The Juila fib benchmark uses the type of function call Julia is good at. Suppose we define fib in a way that uses Python features. def fib(n, dummy): if n >= 2: return fib(n=n-1, dummy=dummy) + fib(dummy=dummy, n=n-2) elif n >= 0: return 1 else: return None, n # or some error indicator including the bad value If the language does not support 'parameter=value' syntax (which existed long before Python), use ('n', n) and ('dummy', dummy) instead. Now let's compare call times. Or lets try 'fib(n-1, dummy) + fib(dummy=dummy, n=n-2)' to compare functions that can identify argments either by position or by name. Or f(n-1, dummy) + f(dummy=dummy, n=n-2) + f(*(n-3, dummy)), and change '2' to '3', to utilize another Python call feature. If the language does not support *args, omit '*'. -- Terry Jan Reedy From robertvstepp at gmail.com Sat Feb 24 02:05:40 2018 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 24 Feb 2018 01:05:40 -0600 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <20180223190916.GG10273@bladeshadow.org> Message-ID: On Fri, Feb 23, 2018 at 1:16 PM, Chris Angelico wrote: > On Sat, Feb 24, 2018 at 6:09 AM, Python wrote: >> On Sat, Feb 24, 2018 at 05:56:25AM +1100, Chris Angelico wrote: >>> No, not satisfied. Everything you've said would still be satisfied if >>> all versions of the benchmark used the same non-recursive algorithm. >>> There's nothing here that says it's testing recursion, just that (for >>> consistency) it's testing the same algorithm. There is no reason to >>> specifically test *recursion*, unless that actually aligns with what >>> you're doing. >> >> It seems abundantly clear to me that testing recursion is the point of >> writing a benchmark implementing recursion (and very little of >> anything else). Again, you can decide for yourself the suitability of >> the benchmark, but I don't think you can really claim it doesn't >> effectively test what it means to. > > Where do you get that it's specifically a recursion benchmark though? > I can't find it anywhere in the explanatory text. I hope I am not missing something blatantly obvious, but in the page Python linked to (https://julialang.org/benchmarks/), in the opening paragraph it states: These micro-benchmarks, while not comprehensive, do test compiler performance on a range of common code patterns, such as function calls, string parsing, sorting, numerical loops, random number generation, recursion, and array operations. Recursion is listed above as one code pattern to specifically target with one of their micro-benchmarks. I did not exhaustively go through each language's code examples, but it does appear that the authors of these benchmarks are implementing as exactly as possible the same algorithm chosen for each of these code patterns in each language. Now to me the real question is whether the Julia people were trying to be intellectually honest in their choices of coding patterns and algorithms or were they deliberately cherry picking these to make Julia look better than the competition? Myself, I would rather be charitable than accusatory about the benchmarkers' intentions. For instance, the authors were aware of numpy and used it for some of the python coding -- the array operations they were targeting IIRC. Instead, if they were being deliberately dishonest, they could have come up with some really contrived python code. -- boB From bc at freeuk.com Sat Feb 24 06:35:17 2018 From: bc at freeuk.com (bartc) Date: Sat, 24 Feb 2018 11:35:17 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> Message-ID: On 24/02/2018 02:46, Steven D'Aprano wrote: > Take the Fibonacci double-recursion benchmark. Okay, it tests how well > your language does at making millions of function calls. Why? How often > do you make millions of function calls? Very often. Unless you are writing code 1970s style with everything in one big main function. I've done some tests with my interpreter [sorry, Ned], on one real task: Total number of byte-code calls: 3.2 million Total number of real x64 calls: 520 million On this specific benchmark: 48 million and 580 million. Changing the way those x64 calls were made (using a different call convention), made some byte-code programs take up to 50% longer to execute. [In this interpreter, each byte-code, no matter what it is, is dispatched via a function call.] For most application code, > executing the function is far more costly than the overhead of calling > it, and the call overhead is dwarfed by the rest of the application. Any actual figures? In the case of interpreters, you want to optimise each byte-code, and one way of doing that is to write a small program which features that byte-code heavily. And then you start tweaking. It is true that when working with heavy-duty data, or offloading work to external, non-byte-code functions, then the byte-code execution overheads are small. But Python's weakness is when it /is/ executing actual algorithms using actual byte-code. And actually, performance of CPython does seem to have improved tremendously over the years. So some people have their eye on the ball. Clearly not you. > If you have a language with tail recursion elimination, you can bet > that's its benchmarks will include examples of tail recursion and tail > recursion will be a favoured idiom in that language. If it doesn't, it > won't. Benchmarks need to be honest. But Fibonacci I think can't use that optimisation (although gcc seems to have found another way of not that much work). -- bartc From bc at freeuk.com Sat Feb 24 09:38:41 2018 From: bc at freeuk.com (bartc) Date: Sat, 24 Feb 2018 14:38:41 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <87bmgfyftm.fsf@bsb.me.uk> Message-ID: On 24/02/2018 02:05, Steven D'Aprano wrote: > On Fri, 23 Feb 2018 19:25:35 +0000, bartc wrote: >> Python is 10 times slower than a competitor = doesn't matter >> My language is 1.5 times slower than the big boys' = matters >> a great deal > As for Python's order-of-magnitude speed difference, thank you for being > generous. Actually that comparison was with a competitor, ie. another dynamic language, because I understand such languages work in different fields from the Cs and C++s. I'm sure there must be some that are faster (years since I've looked at the field), but I vaguely had in mind mine. Although since then, CPython has gotten faster. Note that there are JIT-based implementations now which can give very good results (other than PyPy) with dynamic languages. My own efforts are still byte-code based so are unlikely to get any faster. But they are also very simple. > So it is quite possible to get practical work done and be a competitive, > useful language despite being (allegedly) a thousand or more times slower > than C. Of course. I've been using a dynamic scripting language as an adjunct to my compiled applications since the mid 80s. Then they were crude and hopelessly slow (and machines were a lot slower too), but they could still be tremendously useful with the right balance. But the faster they are, the more work they can take over. -- bartc From pengyu.ut at gmail.com Sat Feb 24 11:17:35 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Sat, 24 Feb 2018 10:17:35 -0600 Subject: read Unicode characters one by one in python2 Message-ID: Here shows some code for reading Unicode characters one by one in python2. Is it the best code for reading Unicode characters one by one in python2? https://rosettacode.org/wiki/Read_a_file_character_by_character/UTF8#Python -- Regards, Peng From phd at phdru.name Sat Feb 24 11:29:33 2018 From: phd at phdru.name (Oleg Broytman) Date: Sat, 24 Feb 2018 17:29:33 +0100 Subject: SQLObject 3.6.0 Message-ID: <20180224162933.GA7274@phdru.name> Hello! I'm pleased to announce version 3.6.0, the first stable release of branch 3.6 of SQLObject. What's new in SQLObject ======================= Contributor for this release is Michael S. Root. Minor features -------------- * Close cursors after using to free resources immediately instead of waiting for gc. Bug fixes --------- * Fix for TypeError using selectBy on a BLOBCol. PR by Michael S. Root. Drivers ------- * Extend support for oursql and Python 3 (requires our fork of the driver). * Fix cursor.arraysize - pymssql doesn't have arraysize. * Set timeout for ODBC with MSSQL. * Fix _setAutoCommit for MSSQL. Documentation ------------- * Document extras that are available for installation. Build ----- * Use ``python_version`` environment marker in ``setup.py`` to make ``install_requires`` and ``extras_require`` declarative. This makes the universal wheel truly universal. * Use ``python_requires`` keyword in ``setup.py``. For a more complete list, please see the news: http://sqlobject.org/News.html What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Python 2.7 or 3.4+ is required. Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Download: https://pypi.python.org/pypi/SQLObject/3.6.0 News and changes: http://sqlobject.org/News.html StackOverflow: https://stackoverflow.com/questions/tagged/sqlobject Example ======= Create a simple class that wraps a table:: >>> from sqlobject import * >>> >>> sqlhub.processConnection = connectionForURI('sqlite:/:memory:') >>> >>> class Person(SQLObject): ... fname = StringCol() ... mi = StringCol(length=1, default=None) ... lname = StringCol() ... >>> Person.createTable() Use the object:: >>> p = Person(fname="John", lname="Doe") >>> p >>> p.fname 'John' >>> p.mi = 'Q' >>> p2 = Person.get(1) >>> p2 >>> p is p2 True Queries:: >>> p3 = Person.selectBy(lname="Doe")[0] >>> p3 >>> pc = Person.select(Person.q.lname=="Doe").count() >>> pc 1 Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From pengyu.ut at gmail.com Sat Feb 24 11:49:25 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Sat, 24 Feb 2018 10:49:25 -0600 Subject: How to extract the raw bytes of the decoded unicode? Message-ID: Hi, I can extracted the encoded value as bytes. But is there a way to extracted the decoded value (for ?, it is C1)? Thanks. $ cat ./dumpunicode.py #!/usr/bin/env python3 while True: c = sys.stdin.read(1) if c: print(c) print('0x' + ''.join(['%x' % x for x in reversed(bytes(c, encoding='utf-8'))])) else: break $ ./dumpunicode.py <<< ? ? 0xa1c3 0xa -- Regards, Peng From Karsten.Hilbert at gmx.net Sat Feb 24 12:31:11 2018 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Sat, 24 Feb 2018 18:31:11 +0100 Subject: read Unicode characters one by one in python2 In-Reply-To: References: Message-ID: <20180224173111.GE915@hermes.hilbert.loc> On Sat, Feb 24, 2018 at 10:17:35AM -0600, Peng Yu wrote: > Here shows some code for reading Unicode characters one by one in > python2. Is it the best code for reading Unicode characters one by one > in python2? > > https://rosettacode.org/wiki/Read_a_file_character_by_character/UTF8#Python This seems off, because in Python2 you can also specify an encoding. However, you seem to be asking about reading the console which comes with more caveats. So, what is it ? Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From pengyu.ut at gmail.com Sat Feb 24 12:41:32 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Sat, 24 Feb 2018 11:41:32 -0600 Subject: How to only get \n for newline without the single quotes? Message-ID: I would like to just get the escaped string without the single quotes. Is there a way to do so? Thanks. >>> x='\n' >>> print repr(x) '\n' -- Regards, Peng From best_lay at yahoo.com Sat Feb 24 13:45:44 2018 From: best_lay at yahoo.com (Wildman) Date: Sat, 24 Feb 2018 12:45:44 -0600 Subject: How to only get \n for newline without the single quotes? References: Message-ID: On Sat, 24 Feb 2018 11:41:32 -0600, Peng Yu wrote: > I would like to just get the escaped string without the single quotes. > Is there a way to do so? Thanks. > >>>> x='\n' >>>> print repr(x) > '\n' Python 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118] on linux Type "help", "copyright", "credits" or "license" for more information. >>> x='/n' >>> print(repr(x)) '/n' >>> print(repr(x).strip("'")) /n >>> Python 2.7.13 (default, Nov 24 2017, 17:33:09) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> x='/n' >>> print repr(x) '/n' >>> print repr(x).strip("'") /n >>> -- GNU/Linux user #557453 "There are only 10 types of people in the world... those who understand Binary... and those who don't." -Spike From pengyu.ut at gmail.com Sat Feb 24 14:08:34 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Sat, 24 Feb 2018 13:08:34 -0600 Subject: How to only get \n for newline without the single quotes? In-Reply-To: References: Message-ID: On Sat, Feb 24, 2018 at 12:45 PM, Wildman via Python-list wrote: > On Sat, 24 Feb 2018 11:41:32 -0600, Peng Yu wrote: > >> I would like to just get the escaped string without the single quotes. >> Is there a way to do so? Thanks. >> >>>>> x='\n' >>>>> print repr(x) >> '\n' > > Python 3.5.3 (default, Jan 19 2017, 14:11:04) > [GCC 6.3.0 20170118] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> x='/n' >>>> print(repr(x)) > '/n' >>>> print(repr(x).strip("'")) > /n >>>> > > Python 2.7.13 (default, Nov 24 2017, 17:33:09) > [GCC 6.3.0 20170516] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> x='/n' >>>> print repr(x) > '/n' >>>> print repr(x).strip("'") > /n >>>> I was looking for something builtin python. There is not such a builtin way? -- Regards, Peng From pengyu.ut at gmail.com Sat Feb 24 14:16:02 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Sat, 24 Feb 2018 13:16:02 -0600 Subject: How to only get \n for newline without the single quotes? In-Reply-To: References: Message-ID: On Sat, Feb 24, 2018 at 1:08 PM, Peng Yu wrote: > On Sat, Feb 24, 2018 at 12:45 PM, Wildman via Python-list > wrote: >> On Sat, 24 Feb 2018 11:41:32 -0600, Peng Yu wrote: >> >>> I would like to just get the escaped string without the single quotes. >>> Is there a way to do so? Thanks. >>> >>>>>> x='\n' >>>>>> print repr(x) >>> '\n' >> >> Python 3.5.3 (default, Jan 19 2017, 14:11:04) >> [GCC 6.3.0 20170118] on linux >> Type "help", "copyright", "credits" or "license" for more information. >>>>> x='/n' >>>>> print(repr(x)) >> '/n' >>>>> print(repr(x).strip("'")) >> /n >>>>> >> >> Python 2.7.13 (default, Nov 24 2017, 17:33:09) >> [GCC 6.3.0 20170516] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> x='/n' >>>>> print repr(x) >> '/n' >>>>> print repr(x).strip("'") >> /n >>>>> > > I was looking for something builtin python. There is not such a builtin way? Also, this is not printed as \f or \e. I'd like things like \a, \b, ... \v be printed as such. Is there a way to do so in python? >>> print repr('\f') '\x0c' >>> print repr('\e') '\\e' -- Regards, Peng From ned at nedbatchelder.com Sat Feb 24 14:17:03 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 24 Feb 2018 14:17:03 -0500 Subject: How to only get \n for newline without the single quotes? In-Reply-To: References: Message-ID: <0d113df2-5e3e-4cdd-4357-d0aa90b1cd3c@nedbatchelder.com> On 2/24/18 2:08 PM, Peng Yu wrote: > On Sat, Feb 24, 2018 at 12:45 PM, Wildman via Python-list > wrote: >> On Sat, 24 Feb 2018 11:41:32 -0600, Peng Yu wrote: >> >>> I would like to just get the escaped string without the single quotes. >>> Is there a way to do so? Thanks. >>> >>>>>> x='\n' >>>>>> print repr(x) >>> '\n' >> Python 3.5.3 (default, Jan 19 2017, 14:11:04) >> [GCC 6.3.0 20170118] on linux >> Type "help", "copyright", "credits" or "license" for more information. >>>>> x='/n' >>>>> print(repr(x)) >> '/n' >>>>> print(repr(x).strip("'")) >> /n >> Python 2.7.13 (default, Nov 24 2017, 17:33:09) >> [GCC 6.3.0 20170516] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> x='/n' >>>>> print repr(x) >> '/n' >>>>> print repr(x).strip("'") >> /n > I was looking for something builtin python. There is not such a builtin way? > Peng, your last three questions all make me wonder what larger problem you are working on.? These all sound like there might be an easier way.? Can you take a step back and tell us about the big picture? --Ned. From kwpolska at gmail.com Sun Feb 25 08:33:33 2018 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 25 Feb 2018 14:33:33 +0100 Subject: read Unicode characters one by one in python2 In-Reply-To: References: Message-ID: On 24 February 2018 at 17:17, Peng Yu wrote: > Here shows some code for reading Unicode characters one by one in > python2. Is it the best code for reading Unicode characters one by one > in python2? > > https://rosettacode.org/wiki/Read_a_file_character_by_character/UTF8#Python No, it?s terrible. So is the Python 3 version. All you need for both Pythons is this: import io with io.open('input.txt', 'r', encoding='utf-8') as fh: for character in fh: print(character) (and please make sure you need to read character-by-character first) -- Chris Warrick PGP: 5EAAEA16 From rosuav at gmail.com Sun Feb 25 09:50:16 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 26 Feb 2018 01:50:16 +1100 Subject: read Unicode characters one by one in python2 In-Reply-To: References: Message-ID: On Mon, Feb 26, 2018 at 12:33 AM, Chris Warrick wrote: > On 24 February 2018 at 17:17, Peng Yu wrote: >> Here shows some code for reading Unicode characters one by one in >> python2. Is it the best code for reading Unicode characters one by one >> in python2? >> >> https://rosettacode.org/wiki/Read_a_file_character_by_character/UTF8#Python > > No, it?s terrible. So is the Python 3 version. All you need for both > Pythons is this: > > import io > with io.open('input.txt', 'r', encoding='utf-8') as fh: > for character in fh: > print(character) If you actually need character-by-character, you'd need "for character in fh.read()" rather than iterating over the file itself. Iterating over a file yields lines. (BTW, if you know for sure that you're running in Python 3, "io.open" can be shorthanded to just "open". They're the same thing in Py3.) ChrisA From steve+comp.lang.python at pearwood.info Sun Feb 25 11:57:58 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 25 Feb 2018 16:57:58 +0000 (UTC) Subject: read Unicode characters one by one in python2 References: Message-ID: On Mon, 26 Feb 2018 01:50:16 +1100, Chris Angelico wrote: > If you actually need character-by-character, you'd need "for character > in fh.read()" rather than iterating over the file itself. Iterating over > a file yields lines. Indeed. But I wonder if there's a performance cost/gain to iterating over each line, rather than reading one char at a time? for line in file: for c in line: ... Too lazy to actually test it myself, but just tossing this idea out in case anyone else cares to give it a try. -- Steve From spluque at gmail.com Sun Feb 25 12:45:51 2018 From: spluque at gmail.com (Seb) Date: Sun, 25 Feb 2018 11:45:51 -0600 Subject: matrix multiplication Message-ID: <87lgfhnesw.fsf@otaria.sebmel.org> Hello, The following is an example of an Nx3 matrix (`uvw`) representing N vectors that need to be multiplied by a 3x3 matrix (generated by `randint_mat` function) and store the result in `uvw_rots`: ---<--------------------cut here---------------start------------------->--- import numpy as np def randint_mat(x): return np.random.randint(x, size=(3, 3)) np.random.seed(123) uvw = np.random.randn(1000, 3) maxint = np.random.randint(1, 10, size=uvw.shape[0]) uvw_rots = np.empty_like(uvw) for i, v in enumerate(maxint): mati = randint_mat(v) uvw_roti = np.dot(uvw[i], mati) uvw_rots[i] = uvw_roti ---<--------------------cut here---------------end--------------------->--- Is there a better (faster) approach than looping through the rows of uvw as shown? Thanks, -- Seb From arya.kumar2494 at gmail.com Sun Feb 25 13:19:43 2018 From: arya.kumar2494 at gmail.com (arya.kumar2494 at gmail.com) Date: Sun, 25 Feb 2018 10:19:43 -0800 (PST) Subject: For Loop Dilema [python-list] Message-ID: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Why we don?t use: for _ in _ in _ Instead of for _ in _: for _ in _: Ex: Names = ["Arya","Pupun"] for name in Names: for c in name: print(c) instead use: for c in name in Names: print(c) From rosuav at gmail.com Sun Feb 25 15:56:44 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 26 Feb 2018 07:56:44 +1100 Subject: read Unicode characters one by one in python2 In-Reply-To: References: Message-ID: On Mon, Feb 26, 2018 at 3:57 AM, Steven D'Aprano wrote: > On Mon, 26 Feb 2018 01:50:16 +1100, Chris Angelico wrote: > >> If you actually need character-by-character, you'd need "for character >> in fh.read()" rather than iterating over the file itself. Iterating over >> a file yields lines. > > Indeed. But I wonder if there's a performance cost/gain to iterating over > each line, rather than reading one char at a time? > > for line in file: > for c in line: > ... > > Too lazy to actually test it myself, but just tossing this idea out in > case anyone else cares to give it a try. > Depends on the size of the file. For a small file, you could read the whole thing into memory in a single disk operation, and then splitting into lines is a waste of time; but for a gigantic file, reading everything into RAM means crazy-expensive transfer/copy, so it'd be HEAPS more efficient to work line by line - particularly if you don't need the whole file. But if you indeed want to cut the process off, having nested loops means a simple "break" won't work. So that's a different consideration. ChrisA From rosuav at gmail.com Sun Feb 25 15:58:58 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 26 Feb 2018 07:58:58 +1100 Subject: For Loop Dilema [python-list] In-Reply-To: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Message-ID: On Mon, Feb 26, 2018 at 5:19 AM, wrote: > Why we don?t use: > > for _ in _ in _ > > Instead of > > for _ in _: > for _ in _: > > Ex: > > Names = ["Arya","Pupun"] > > for name in Names: > for c in name: > print(c) > > instead use: > > for c in name in Names: > print(c) Because programming is all about building up a program from primitives. The number of times when we need this kind of nested loop (with absolutely nothing in between the loops) is way too small to justify dedicated syntax. ChrisA From tjreedy at udel.edu Sun Feb 25 18:52:14 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 25 Feb 2018 18:52:14 -0500 Subject: matrix multiplication In-Reply-To: <87lgfhnesw.fsf@otaria.sebmel.org> References: <87lgfhnesw.fsf@otaria.sebmel.org> Message-ID: On 2/25/2018 12:45 PM, Seb wrote: > Hello, > > The following is an example of an Nx3 matrix (`uvw`) representing N > vectors that need to be multiplied by a 3x3 matrix (generated by > `randint_mat` function) and store the result in `uvw_rots`: > > ---<--------------------cut here---------------start------------------->--- > import numpy as np > > > def randint_mat(x): > return np.random.randint(x, size=(3, 3)) > > > np.random.seed(123) > uvw = np.random.randn(1000, 3) > maxint = np.random.randint(1, 10, size=uvw.shape[0]) > > uvw_rots = np.empty_like(uvw) > for i, v in enumerate(maxint): > mati = randint_mat(v) > uvw_roti = np.dot(uvw[i], mati) > uvw_rots[i] = uvw_roti > ---<--------------------cut here---------------end--------------------->--- > > Is there a better (faster) approach than looping through the rows of uvw > as shown? numpy has a matrix multiply function and now the '@' matrix multiply operator. -- Terry Jan Reedy From ian.g.kelly at gmail.com Sun Feb 25 19:49:05 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 25 Feb 2018 17:49:05 -0700 Subject: For Loop Dilema [python-list] In-Reply-To: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Message-ID: On Sun, Feb 25, 2018 at 11:19 AM, wrote: > Why we don?t use: > > for _ in _ in _ > > Instead of > > for _ in _: > for _ in _: > > Ex: > > Names = ["Arya","Pupun"] > > for name in Names: > for c in name: > print(c) > > instead use: > > for c in name in Names: > print(c) It doesn't seem very intuitive (doesn't follow proper English phrasing, for instance) and I don't think it's a common enough situation to warrant adding a special syntax for it. But if you really want it, you could use something like this: def double_for(iterable): for outer in iterable: yield from outer for c in double_for(Names): print(c) But I don't think this is any clearer than making the loops explicit. From rantingrickjohnson at gmail.com Sun Feb 25 19:52:53 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 25 Feb 2018 16:52:53 -0800 (PST) Subject: For Loop Dilema [python-list] In-Reply-To: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Message-ID: <7690174c-8a58-4bdd-accd-4410508f4cc1@googlegroups.com> On Sunday, February 25, 2018 at 12:19:56 PM UTC-6, arya.ku... at gmail.com wrote: > Ex: > > Names = ["Arya","Pupun"] > > for name in Names: > for c in name: > print(c) > > instead use: > > for c in name in Names: > print(c) Hmm. Why stop there? bit = ["kibbles"] bits = [bit, bit] bitts = [bits, bits] for kibbles in bit in bits in bitts: do_something(kibbles) From rantingrickjohnson at gmail.com Sun Feb 25 21:33:47 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 25 Feb 2018 18:33:47 -0800 (PST) Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> On Friday, February 23, 2018 at 10:41:45 AM UTC-6, Steven D'Aprano wrote: [...] > There are dozens of languages that have made the design > choice to limit their default integers to 16- 32- or 64-bit > fixed size, and let the user worry about overflow. Bart, > why does it upset you so that Python made a different > choice? A default "integer-diversity-and-inclusivity-doctrine" is all fine and dandy by me, (Hey, even integers need safe spaces), but i do wish we pythonistas had a method to turn off this (and other) cycle burning "features" -- you know -- in the 99.99999 percent of time that we don't need them. And BTW... Am i the *ONLY* person here who feels that Python optimizations should be more than merely the tossing of dead weight overboard? From rosuav at gmail.com Sun Feb 25 21:45:22 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 26 Feb 2018 13:45:22 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Mon, Feb 26, 2018 at 1:33 PM, Rick Johnson wrote: > On Friday, February 23, 2018 at 10:41:45 AM UTC-6, Steven D'Aprano wrote: > [...] >> There are dozens of languages that have made the design >> choice to limit their default integers to 16- 32- or 64-bit >> fixed size, and let the user worry about overflow. Bart, >> why does it upset you so that Python made a different >> choice? > > A default "integer-diversity-and-inclusivity-doctrine" is > all fine and dandy by me, (Hey, even integers need safe spaces), In Python 3.6+, integers have safe underscores instead. > but i do wish we pythonistas had a method to turn off this > (and other) cycle burning "features" -- you know -- in the > 99.99999 percent of time that we don't need them. Definitely. We should also provide a way for people to manually allocate memory, for the times when that would be more efficient. And to switch out the syntax to include braces. And all those other things people like from other languages. Hey, here's an idea: maybe Python shouldn't stop people from using other languages, if those other languages are better suited to the job? > And BTW... Am i the *ONLY* person here who feels that Python > optimizations should be more than merely the tossing of dead > weight overboard? I dunno. We optimize this mailing list that way...... *ducking for cover* ChrisA From songofacandy at gmail.com Sun Feb 25 22:05:21 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Mon, 26 Feb 2018 12:05:21 +0900 Subject: For Loop Dilema [python-list] In-Reply-To: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Message-ID: https://docs.python.org/3.6/library/itertools.html#itertools.product On Mon, Feb 26, 2018 at 3:19 AM, wrote: > Why we don?t use: > > for _ in _ in _ > > Instead of > > for _ in _: > for _ in _: > > Ex: > > Names = ["Arya","Pupun"] > > for name in Names: > for c in name: > print(c) > > instead use: > > for c in name in Names: > print(c) > -- > https://mail.python.org/mailman/listinfo/python-list -- INADA Naoki From rantingrickjohnson at gmail.com Sun Feb 25 22:26:12 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 25 Feb 2018 19:26:12 -0800 (PST) Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> Message-ID: <700aebe6-0d27-43c0-a05c-f64ad75e50f3@googlegroups.com> On Friday, February 23, 2018 at 8:48:55 PM UTC-6, Steven D'Aprano wrote: [...] > Take the Fibonacci double-recursion benchmark. Okay, it > tests how well your language does at making millions of > function calls. Why? Because making "millons of function calls" is what software *DOES*! Granted, and as others have testified in this thread, i myself have almost never implemented an algorithm like that monstrosity of recursion that is implemented in fib(), but whether we use recursive functions are not, a high number of functions calls is inevitable in our software. And if you don't believe me, then use sys.setrace(myFunc) to trace all function calls to stdout, and you shall be enlightened[1]. > How often do you make millions of function calls? All the time! Of course, if the extent of your code base is a single script containing the source `print 'Hello World'`, perhaps your experience may vary from others here. > For most application code, executing the function is far > more costly than the overhead of calling it, What? Your functions are only called _once_ during the lifetime of your code execution? Enough with the leg pulling. > and the call overhead is dwarfed by the rest of the > application. Of course, because it's calling _other_ functions. > For many, many applications, the *entire* program run could > take orders of magnitude fewer function calls than a single > call to fib(38). Hmm, let's see: def main(): print 'hello world' main() By golly, he's right! O_O -- [1] But, in the meantime, you may want to make yourself a taco; go play a round of golf; and possibly pay a visit to national monument (or three!), while all that crap is printing to stdout. From rantingrickjohnson at gmail.com Sun Feb 25 23:22:17 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 25 Feb 2018 20:22:17 -0800 (PST) Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Sunday, February 25, 2018 at 8:45:56 PM UTC-6, Chris Angelico wrote: > On Mon, Feb 26, 2018 at 1:33 PM, Rick Johnson [...] > > but i do wish we pythonistas had a method to turn off this > > (and other) cycle burning "features" -- you know -- in the > > 99.99999 percent of time that we don't need them. > > Definitely. We should also provide a way for people to > manually allocate memory, for the times when that would be > more efficient. And to switch out the syntax to include > braces. Now you're throwing the baby out with the bath water. We needn't re-implement the C language simply to achieve reasonable and practical code-execution efficiency. Python was never intended to be the fastest language. Heck! Python was never intented to be the fastest _scripting_ language! So of course, speed is not and should not be the primary concern, but to say that execution speed is of _no_ concern is quite absurd indeed. As Steven mentioned, few people, even of they _could_ afford the absurd price, will ever purchase that high-end sports car. However, by using this example Steve is committing the argumentation sin of comparing apples to oranges because transportation is more a matter of practicality. Sure, you may arrive at your destination more quickly in the high-end sports car, however, the trade-off will be that you could be thrown in jail for speeding or be injured or killed in an accident. But such existential risks are not the case for software... Code that execute more quickly simply (wait for it...) yes, executes more quickly! And I can assure you: no one will die, be injured, or even become "Bubba's" cell mate should their code commit the unforgivable "sin" of executing more quickly[1]. -- [1] Quick! Someone please call a priest! We need to confess!!! From rantingrickjohnson at gmail.com Sun Feb 25 23:25:42 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 25 Feb 2018 20:25:42 -0800 (PST) Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: <5de65e9b-790d-42db-a289-6b10fbb53b33@googlegroups.com> On Sunday, February 25, 2018 at 8:45:56 PM UTC-6, Chris Angelico wrote: > On Mon, Feb 26, 2018 at 1:33 PM, Rick Johnson > wrote: [...] > > A default "integer-diversity-and-inclusivity-doctrine" is > > all fine and dandy by me, (Hey, even integers need safe spaces), > > In Python 3.6+, integers have safe underscores instead. You spacist!!! From steve+comp.lang.python at pearwood.info Sun Feb 25 23:32:34 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 26 Feb 2018 04:32:34 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Sun, 25 Feb 2018 18:33:47 -0800, Rick Johnson wrote: > On Friday, February 23, 2018 at 10:41:45 AM UTC-6, Steven D'Aprano > wrote: [...] >> There are dozens of languages that have made the design choice to limit >> their default integers to 16- 32- or 64-bit fixed size, and let the >> user worry about overflow. Bart, why does it upset you so that Python >> made a different choice? > > A default "integer-diversity-and-inclusivity-doctrine" is all fine and > dandy by me, (Hey, even integers need safe spaces), but i do wish we > pythonistas had a method to turn off this (and other) cycle burning > "features" -- you know -- in the 99.99999 percent of time that we don't > need them. Ah, you mean just like the way things were in Python 1.0 through 2.1? Hands up anyone who has seen an integer OverflowError in the last 10 years? Anyone? [steve at ando ~]$ python1.5 -c "print 2**64" Traceback (innermost last): File "", line 1, in ? OverflowError: integer pow() I really miss having to either add, or delete, an "L" suffix from my long ints, and having to catch OverflowError to get any work done, and generally spending half my time worrying how my algorithms will behave when integer operations overflow. Good times. I miss those days. I also miss the days when everyone had scabies. As someone who wrote Python code when bignums where *not* the default, I can tell you that: - it was a real PITA for those who cared about their code working correctly and being bug-free; - and the speed up actually wasn't that meaningful. As is so often the case with these things, using fixed size ints looks good in benchmark games, but what's fast in a toy benchmark and what's fast in real code are not always the same. -- Steve From rantingrickjohnson at gmail.com Mon Feb 26 00:19:19 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 25 Feb 2018 21:19:19 -0800 (PST) Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Sunday, February 25, 2018 at 10:35:29 PM UTC-6, Steven D'Aprano wrote: [...] > Ah, you mean just like the way things were in Python 1.0 > through 2.1? Hands up anyone who has seen an integer > OverflowError in the last 10 years? Anyone? I think Python2.1 is much older than 10 years, so yeah, of course almost no one (save a few old timers) have seen that error. :-) > [...] > I really miss having to either add, or delete, an "L" > suffix from my long ints, and having to catch OverflowError > to get any work done, and generally spending half my time > worrying how my algorithms will behave when integer > operations overflow. I agree with your sarcasm. And that's why these types of auto conversions should be optional. I agree that most times it's more practical to let python handle the dirty details. But in some cases, where you need to squeeze out a few more drops of speed juice, you won't mind the extra trouble. And perhaps you (meaning specifically you) will never need such a flexible feature. But hey. The Python community is diverse. So please do keep that in mind. From steve+comp.lang.python at pearwood.info Mon Feb 26 04:22:57 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 26 Feb 2018 09:22:57 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Sun, 25 Feb 2018 21:19:19 -0800, Rick Johnson wrote: > I agree with your sarcasm. And that's why these types of auto > conversions should be optional. I agree that most times it's more > practical to let python handle the dirty details. But in some cases, > where you need to squeeze out a few more drops of speed juice, you won't > mind the extra trouble. And that's where you call out to a library like numpy, or write a C extension, or use a tool like Numba or Cython to optimise your Python code to use native ints. (Numba is still a work in progress, but Cython is a mature working product.) Or to put it another way... if you want machine ints in Python, the way you get them is something like: from numba import jit @jit def myfunction(): ... The core language doesn't have to support these things when there is a healthy language ecosystem that can do it. -- Steve From sotaro237 at gmail.com Mon Feb 26 04:40:16 2018 From: sotaro237 at gmail.com (sotaro237 at gmail.com) Date: Mon, 26 Feb 2018 01:40:16 -0800 (PST) Subject: help me ? Message-ID: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Define 2 lists. The first one must contain the integer values 1, 2 and 3 and the second one the string values a, b and c. Iterate through both lists to create another list that contains all the combinations of the A and B elements. The final list should look like one of the 2 lists: 1. [1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c] 2. [a1, a2. a3, b1, b2, b3, c1, c2, c3] BONUS: Make the final list contain all possible combinations : [1a, a1, 1b, b1, 1c, c1, 2a, a2, 2b, b2, 2c, c2, 3a, a3, 3b, b3, 3c, c3] Help me ! From auriocus at gmx.de Mon Feb 26 04:51:45 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Mon, 26 Feb 2018 10:51:45 +0100 Subject: help me ? In-Reply-To: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: Am 26.02.18 um 10:40 schrieb sotaro237 at gmail.com: > Define 2 lists. The first one must contain the integer values 1, 2 and 3 and the second one the string values a, b and c. Iterate through both lists to create another list that contains all the combinations of the A and B elements. The final list should look like one of the 2 lists: > 1. [1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c Look up list comprehensions. Christian From steve+comp.lang.python at pearwood.info Mon Feb 26 04:57:06 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 26 Feb 2018 09:57:06 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Sun, 25 Feb 2018 20:22:17 -0800, Rick Johnson wrote: > So of course, speed is not and should not be the > primary concern, but to say that execution speed is of _no_ concern is > quite absurd indeed. I'm pretty sure that nobody here has said that speed is of no concern. Rather, I would argue that the position we're taking is that *in general* Python is fast enough for the sorts of tasks we use it for (except, of course, when it isn't, in which case you have our sympathy, and if we can suggest some solutions we will). Of course we'd take speed optimizations if they were free, but they're never free: - they often require more memory to run; - they usually require more code, which adds to the maintenance burden and increases the interpreter bloat; - and increases the risk of bugs; - somebody has to write, debug, document and maintain the code, and developer time and effort is in short supply; - or the optimization requires changes to the way Python operates; - even if we wanted to make that change, it will break backwards compatibility; - and often what people imagine is a simple optimization (because they haven't tried it) isn't simple at all; - or simply doesn't work; - and most importantly, just saying "Make it go faster!" doesn't work, we actually need to care about the details of *how* to make it faster. (We tried painting Go Faster stripes on the server, and it didn't work.) There's no point suggesting such major changes to Python that requires going back to the drawing board, to Python 0.1 or earlier, and changing the entire execution and memory model of the language. That would just mean we've swapped from a successful, solid, reliable version 3.6 of the language to an untried, unstable, unproven, bug-ridden version 0.1 of New Python. And like New Coke, it won't attract new users (they're already using Javascript or Go or whatever...) and it will alienate existing users (if they wanted Javascript or Go they'd already be using it). There have been at least two (maybe three) attempts to remove the GIL from CPython. They've all turned out to increase complexity by a huge amount, and not actually provide the hoped-for speed increase. Under many common scenarios, the GIL-less CPython actually ran *slower*. (I say "hoped for", but it was more wishful thinking than a rational expectation that removing the GIL would speed things up. I don't believe any of the core developers were surprised that removing the GIL didn't increase speeds much, if at all, and sometimes slowed things down. The believe that the GIL slows Python down is mostly due to a really simplistic understanding of how threading works.) Besides, if you want Python with no GIL so you can run threaded code, why aren't you using IronPython or Jython? Another example: UnladenSwallow. That (mostly) failed to meet expectations because the compiler tool chain being used wasn't mature enough. If somebody were to re-do the project again now, the results might be different. But it turns out that not that many people care enough to make Python faster to actually invest money in it. Everyone wants to just stamp their foot and get it for free. (If it weren't for the EU government funding it, PyPy would probably be languishing in oblivion. Everyone wants a faster Python so long as they don't have to pay for it, or do the work.) A third example: Victor Stinner's FatPython, which seemed really promising in theory, but turned out to not make enough progress fast enough and he lost interest. I have mentioned FatPython here a number of times. All you people who complain that Python is "too slow" and that the core devs ought to do something about it... did any of you volunteer any time to the FatPython project? -- Steve From lutz.horn at posteo.de Mon Feb 26 05:05:45 2018 From: lutz.horn at posteo.de (Lutz Horn) Date: Mon, 26 Feb 2018 11:05:45 +0100 Subject: help me =?UTF-8?Q?=3F?= In-Reply-To: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: > Define 2 lists. ... [...] > Help me ! Sounds like homework. Have you tried anything? Does it work? From tkadm30 at yandex.com Mon Feb 26 05:25:33 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Mon, 26 Feb 2018 05:25:33 -0500 Subject: Detection of ultrasonic side channels in mobile devices with Python? Message-ID: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> Hi, I would like to know if its possible to detect or decode ultrasonic signals in mobiles devices like Android with Python? For a introduction to ultrasonic tracking, see: https://www.wired.com/2016/11/block-ultrasonic-signals-didnt-know-tracking/ Thank you, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From steve+comp.lang.python at pearwood.info Mon Feb 26 05:36:49 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 26 Feb 2018 10:36:49 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <700aebe6-0d27-43c0-a05c-f64ad75e50f3@googlegroups.com> Message-ID: On Sun, 25 Feb 2018 19:26:12 -0800, Rick Johnson wrote: > On Friday, February 23, 2018 at 8:48:55 PM UTC-6, Steven D'Aprano wrote: > [...] >> Take the Fibonacci double-recursion benchmark. Okay, it tests how well >> your language does at making millions of function calls. Why? > > Because making "millons of function calls" is what software *DOES*! You're right -- I worded that badly, and ended up saying something I didn't really mean. Mea culpa. Of course over the course of the entire program run, most non-trivial programmers will end up having millions (if not billions) of function calls *in total*. But they're not all happening *together*. What I tried to get across is, how often do we use millions of function calls to get one value? By default, Python will only allow one thousand function calls in a stack before raising RecursionError. py> def bad(): ... return bad() ... py> bad() Traceback (most recent call last): File "", line 1, in File "", line 2, in bad [ previous line repeats 998 times ] RecursionError: maximum recursion depth exceeded There's nothing unique about recursion that will trigger this error. Any call stack of more than 1000 function calls will do it. So forget about code getting a million functions deep. The average is probably more like a dozen, perhaps a hundred tops. [...] >> For most application code, executing the function is far more costly >> than the overhead of calling it, > > What? Your functions are only called _once_ during the lifetime of your > code execution? Enough with the leg pulling. Let me explain what I intended to say. Here is a really poor estimate of the overhead of calling a function on my computer, measured in iPython. In [50]: def call(): ....: pass ....: In [51]: %timeit call() 1000000 loops, best of 3: 717 ns per loop So, on my computer, in iPython, it takes 717ns to call a function that does nothing. How long does it take to do the tiniest smidgen of useful work? In [52]: def do_something(): ....: a = 1 ....: b = a+1 ....: return b ....: In [53]: %timeit do_something() 1000000 loops, best of 3: 923 ns per loop Only an extra 200 ns! Wow, function calls are expensive!!! (Well, yes, I knew that.) However, let's do some *real* work, not a Mickey Mouse function like do_something(). In [54]: import pyprimes In [55]: %timeit pyprimes.nth_prime(100000) 1 loops, best of 3: 2.6 s per loop I happen to know that calling nth_prime ends up making a couple of hundred function calls. (Because I wrote it.) Let's call it 10,000 function calls, to be on the safe side. So 10,000 x 800 nanoseconds is 0.008 second, let's call it 0.01 second, which is just 0.4% of the total cost of calculating the number I was after. (For the record, that's 1299709.) So just a trivial fraction of the total cost of the function. Now I know that this is not always the case. But I maintain that for *most* (but not all) practical, real-world code that actually does something useful, the overhead of calling the functions is *usually* (but not always) low compared to the cost of actually doing whatever it is that the functions do. -- Steve From __peter__ at web.de Mon Feb 26 05:45:49 2018 From: __peter__ at web.de (Peter Otten) Date: Mon, 26 Feb 2018 11:45:49 +0100 Subject: help me ? References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: Christian Gollwitzer wrote: > Am 26.02.18 um 10:40 schrieb sotaro237 at gmail.com: >> Define 2 lists. The first one must contain the integer values 1, 2 and 3 >> and the second one the string values a, b and c. Iterate through both >> lists to create another list that contains all the combinations of the A >> and B elements. The final list should look like one of the 2 lists: 1. >> [1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c > > Look up list comprehensions. Hm, this will teach the OP a concise way to rewrite the solution once he has solved the problem with the conventional nested for loops, list.append(), and str.format()... From bc at freeuk.com Mon Feb 26 06:13:51 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 11:13:51 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On 26/02/2018 09:22, Steven D'Aprano wrote: > On Sun, 25 Feb 2018 21:19:19 -0800, Rick Johnson wrote: > > >> I agree with your sarcasm. And that's why these types of auto >> conversions should be optional. I agree that most times it's more >> practical to let python handle the dirty details. But in some cases, >> where you need to squeeze out a few more drops of speed juice, you won't >> mind the extra trouble. > > And that's where you call out to a library like numpy, or write a C > extension, or use a tool like Numba or Cython to optimise your Python > code to use native ints. (Numba is still a work in progress, but Cython > is a mature working product.) > > Or to put it another way... if you want machine ints in Python, the way > you get them is something like: > > from numba import jit > > @jit > def myfunction(): ... > > > > The core language doesn't have to support these things when there is a > healthy language ecosystem that can do it. Below is the first draft of a Python port of a program to do with random numbers. (Ported from my language, which in turned ported it from a C program by George Marsaglia, the random number guy.) However, running it quickly exhausts the memory in my machine. The reason is that Python unhelpfully widens all results to bignums as needed. The code relies on calculations being modulo 2^64. Note that restricting integer ops to 64 bits probably still won't work, as I believe the numbers need to be unsigned. ------------------------------------------ Q=0 carry=36243678541 xcng=12367890123456 xs=521288629546311 indx=20632 def refill(): global Q, carry, indx for i in range(20632): h = carry & 1 z = ((Q[i]<<41)>>1)+((Q[i]<<39)>>1)+(carry>>1) carry = (Q[i]>>23)+(Q[i]>>25)+(z>>63) Q[i] = ~((z<<1)+h) indx=1 return Q[0] def start(): global Q, carry, xcng, xs, indx Q=[0,]*20632 for i in range(20632): xcng=6906969069 * xcng + 123 xs ^= (xs<<13) xs ^= (xs>>17) xs ^= (xs<<43) Q[i] = xcng + xs N = 100000 for i in range(N): if indx<20632: s = Q[indx] indx+=1 else: s = refill() xcng=6906969069 * xcng + 123 xs ^= (xs<<13) xs ^= (xs>>17) xs ^= (xs<<43) x = s+xcng+xs print ("Does x= 4013566000157423768") print (" x=",x) start() ------------------------------------------ (The code performs N iterations of a random number generator. You get the result expected, ie. x=401...768, when N is a billion.) -- Bartc From rosuav at gmail.com Mon Feb 26 06:34:00 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 26 Feb 2018 22:34:00 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Mon, Feb 26, 2018 at 8:57 PM, Steven D'Aprano wrote: > On Sun, 25 Feb 2018 20:22:17 -0800, Rick Johnson wrote: > >> So of course, speed is not and should not be the >> primary concern, but to say that execution speed is of _no_ concern is >> quite absurd indeed. > > I'm pretty sure that nobody here has said that speed is of no concern. > > Rather, I would argue that the position we're taking is that *in general* > Python is fast enough for the sorts of tasks we use it for (except, of > course, when it isn't, in which case you have our sympathy, and if we can > suggest some solutions we will). > > Of course we'd take speed optimizations if they were free, but they're > never free: > > - they often require more memory to run; > > - they usually require more code, which adds to the maintenance burden > and increases the interpreter bloat; > > - and increases the risk of bugs; > > - somebody has to write, debug, document and maintain the code, > and developer time and effort is in short supply; > > - or the optimization requires changes to the way Python operates; > > - even if we wanted to make that change, it will break backwards > compatibility; > > - and often what people imagine is a simple optimization (because > they haven't tried it) isn't simple at all; > > - or simply doesn't work; > > - and most importantly, just saying "Make it go faster!" doesn't work, > we actually need to care about the details of *how* to make it faster. Or it reduces memory usage, improves performance, and makes things easier on the programmer... but might place unexpected (or unwanted) demands on other implementations of Python. CPython, by virtue of being the "default" Python interpreter, has to be careful of appearing to mandate something. That's why buy-in from other Python implementation teams (I believe Jython was the most notable here) was important in the discussion about the recent compact dict update. Even what looks like a no-brainer still can have unwanted consequences. > (We tried painting Go Faster stripes on the server, and it didn't work.) Steven Middlename D'Aprano! You should know better than that. "It didn't work" is not good enough. What actually happened? Did the stripes smoulder and smoke until you powered down the server? Did the server raise NotImplementedError when you touched it with the paintbrush? Did your boss walk up behind you and hand you a pink slip? Be specific! > There have been at least two (maybe three) attempts to remove the GIL > from CPython. They've all turned out to increase complexity by a huge > amount, and not actually provide the hoped-for speed increase. Under many > common scenarios, the GIL-less CPython actually ran *slower*. > > (I say "hoped for", but it was more wishful thinking than a rational > expectation that removing the GIL would speed things up. I don't believe > any of the core developers were surprised that removing the GIL didn't > increase speeds much, if at all, and sometimes slowed things down. The > believe that the GIL slows Python down is mostly due to a really > simplistic understanding of how threading works.) Removing the GIL from CPython is not about "speeding up" the language or the interpreter, but about improving parallelism. And I think all the core devs understand this (hence the lack of surprise there), but people who call for it often don't. > (If it weren't for the EU government funding it, PyPy would probably be > languishing in oblivion. Everyone wants a faster Python so long as they > don't have to pay for it, or do the work.) Hm, I didn't know the EU was funding PyPy. Okay. Cool. ChrisA From rosuav at gmail.com Mon Feb 26 06:40:36 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 26 Feb 2018 22:40:36 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Mon, Feb 26, 2018 at 10:13 PM, bartc wrote: > Below is the first draft of a Python port of a program to do with random > numbers. (Ported from my language, which in turned ported it from a C > program by George Marsaglia, the random number guy.) > > However, running it quickly exhausts the memory in my machine. The reason is > that Python unhelpfully widens all results to bignums as needed. The code > relies on calculations being modulo 2^64. > > Note that restricting integer ops to 64 bits probably still won't work, as I > believe the numbers need to be unsigned. No, it's because the original implementation assumed integer wrap-around (at least, I think that's what's happening; I haven't analyzed the code in great detail). That means all your integer operations are doing two jobs: the one they claim to, and then a masking to 64 bits signed. That's two abstract operations that happen, due to the nature of the CPU, to work efficiently together. If you don't implement both halves of that in your Python port, you have failed at porting. What if you were porting a program from a 72-bit chip that assumed Binary Coded Decimal? Would you complain that C's integers are badly behaved? And that's without even asking whether a C program that assumes integer wrap-around counts as portable. At least with Python, you have a guarantee that integer operations are going to behave the same way on all compliant implementations of the language. ChrisA From antoon.pardon at vub.be Mon Feb 26 07:06:51 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Mon, 26 Feb 2018 13:06:51 +0100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 23-02-18 02:27, Steven D'Aprano wrote: > Why do you care about the 50 million calls? That's crazy -- the important > thing is *calculating the Fibonacci numbers as efficiently as possible*. No necessarily. David Beazley in his talks sometimes uses an ineffecient algorithm for calculating fibonacci numbers because he needs something that uses the cpu intensively. calculating the fibonacci numbers in that context as efficiently as possible would defeat that purpose. So in a context of a benchmark it is not unreasonable to assume those 50 million calls are the purpose and not calculating the Fibonacci numbers as efficiently as possible. -- Antoon. ? From bc at freeuk.com Mon Feb 26 07:13:56 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 12:13:56 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On 26/02/2018 11:40, Chris Angelico wrote: > On Mon, Feb 26, 2018 at 10:13 PM, bartc wrote: >> Below is the first draft of a Python port of a program to do with random >> numbers. (Ported from my language, which in turned ported it from a C >> program by George Marsaglia, the random number guy.) >> >> However, running it quickly exhausts the memory in my machine. The reason is >> that Python unhelpfully widens all results to bignums as needed. The code >> relies on calculations being modulo 2^64. >> >> Note that restricting integer ops to 64 bits probably still won't work, as I >> believe the numbers need to be unsigned. > > No, it's because the original implementation assumed integer > wrap-around (at least, I think that's what's happening; I haven't > analyzed the code in great detail). That means all your integer > operations are doing two jobs: the one they claim to, and then a > masking to 64 bits signed. That's two abstract operations that happen, > due to the nature of the CPU, to work efficiently together. If you > don't implement both halves of that in your Python port, you have > failed at porting. What if you were porting a program from a 72-bit > chip that assumed Binary Coded Decimal? Would you complain that C's > integers are badly behaved? > > And that's without even asking whether a C program that assumes > integer wrap-around counts as portable. At least with Python, you have > a guarantee that integer operations are going to behave the same way > on all compliant implementations of the language. A C version is given below. (One I may have messed around with, which I'm not sure works properly. For an original, google for Marsaglia and KISS64 or SUPRKISS64.) Most integers are unsigned, which have well-defined overflow in C (they just wrap as expected). In C, a mixed signed/unsigned op is performed as unsigned. ----------------------------- /* SUPRKISS64.c, period 5*2^1320480*(2^64-1) */ #include #include #include "timer.c" static signed long long Q[20632], carry=36243678541LL, xcng=12367890123456LL, xs=521288629546311LL, indx=20632; #define CNG ( xcng=6906969069LL*xcng+123 ) #define XS ( xs^=xs<<13,xs^=xs>>17,xs^=xs<<43 ) #define SUPR ( indx<20632 ? Q[indx++] : refill() ) #define KISS SUPR+CNG+XS signed long long refill(void) { int i; signed long long z,h; for(i=0;i<20632;i++) { h = (carry&1); z = ((Q[i]<<41)>>1)+((Q[i]<<39)>>1)+(carry>>1); carry = (Q[i]>>23)+(Q[i]>>25)+(z>>63); Q[i] = ~((z<<1)+h); } indx=1; return (Q[0]); } int main() { int i; signed long long x; for(i=0;i<20632;i++) Q[i]=CNG+XS; for(i=0;i<1000000000;i++) x=KISS; printf("Does x=4013566000157423768\n x=%llu.\n",x); } -- bartc From antoon.pardon at vub.be Mon Feb 26 07:28:41 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Mon, 26 Feb 2018 13:28:41 +0100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> Message-ID: <3b778258-91ab-08dd-7b35-768640aa4aad@vub.be> On 24-02-18 06:18, Terry Reedy wrote: > On 2/23/2018 11:32 AM, Python wrote: >> >> Doing that would completely fail to accomplish the task of comparing >> the performance of recursive function calls in the two languages, >> which is what the benchmark was designed to do. > > That is an non goal because *languages* do not have clock-time speeds, > only programs written in concrete implementations run on real hardware. > > Why do you think it fair to compare function call times using the > slowest rather than the fastest implementation of Python function > calls? ?Real Python programmers who are seriously concerned about time > try to use the fastest implementation appropriate to the situation. If the slowest happens to be the standard implementation I see nothing wrong with it. > > >? So, no actually, it shouldn't. > > To me, it is 'not using the fasted Python' that fails to make apples > to apples comparisons. > > It has been said here that comparisons should use the same algorithm > doing the much the same work in both implementation.? However, a > Python function call does *much more work* than in most languages, > because the meaning of 'function call' is much more flexible in Python > than most languages. So? I don't see what is wrong when is pointed out that all that extra work, that a lot of people don't care about is costing performance. > The usual comparison is like lemons (other language calls) to oranges > (Python language calls, much more flexible).? To be fair, the > comparison should be to a Python implementation that either notices or > accepts a declaration that, for instance, fib(n) only needs to pass an > int by position. > > Comparing int addition to python object addition also compares unlike > operations.? To compare using the same addition algorithm, one must > use an implementation that can restrict '+' to just int addition. I don't agree, if the performance loss comes from the standard implementation of the language unable of doing that then people testing languages shouldn't be burdened with going to search for some implementation that can. -- Antoon Pardon From bc at freeuk.com Mon Feb 26 07:29:34 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 12:29:34 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: Message-ID: On 26/02/2018 12:06, Antoon Pardon wrote: > On 23-02-18 02:27, Steven D'Aprano wrote: >> Why do you care about the 50 million calls? That's crazy -- the important >> thing is *calculating the Fibonacci numbers as efficiently as possible*. > > No necessarily. > > David Beazley in his talks sometimes uses an ineffecient algorithm for calculating > fibonacci numbers because he needs something that uses the cpu intensively. > calculating the fibonacci numbers in that context as efficiently as possible would > defeat that purpose. > > So in a context of a benchmark it is not unreasonable to assume those 50 million > calls are the purpose and not calculating the Fibonacci numbers as efficiently as > possible. I don't think Steven is ever going to concede this point. Because Python performs badly compared to Julia or C, and it's not possible to conveniently offload the task to some fast library because it only uses a handful of primitive byte-codes. (I have the same trouble with my own interpreted language. Although somewhat brisker than CPython, it will always be much slower than a C-like language on such micro-benchmarks. But I accept that; I don't have an army of people working on acceleration projects and tracing JIT compilers. To those people however, such a benchmark can be a useful yardstick of progress.) -- bartc From steve+comp.lang.python at pearwood.info Mon Feb 26 08:03:34 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 26 Feb 2018 13:03:34 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Mon, 26 Feb 2018 22:34:00 +1100, Chris Angelico wrote: [...] >> (We tried painting Go Faster stripes on the server, and it didn't >> work.) > > Steven Middlename D'Aprano! You should know better than that. "It didn't > work" is not good enough. What actually happened? A truck crashed into the power pole outside our building and the power went off. Then the UPS ran down and the server crashed. Clearly this is a bug in the go-faster stripes. We tried reporting it: Expected behaviour: server goes faster. Actual behaviour: a truck crashes into the power pole across the street and the lights go out. but the maintainers closed the issue "Works for me". Very disappointed! [...] > Removing the GIL from CPython is not about "speeding up" the language or > the interpreter, but about improving parallelism. It is about speeding up threaded code which is CPU-bound. I call that speeding up the language :-) Hypothetically, it could also speed up even unthreaded code. Some language features could internally launch threads and operate using implicit parallelism. For instance, map(func, seq) could run in parallel in separate threads whenever Python knew that func had no side-effects, say for built-ins. There are also parallel algorithms for bignum arithmetic. If threads were quick enough, I'm sure we could come up with many more examples. Hypothetically speaking. (In practice, the old hope that parallel computing would speed everything up turned out to be flawed. Relatively few tasks are embarrassingly parallel, most have sequential bottlenecks, and many are inherently sequential.) >> (If it weren't for the EU government funding it, PyPy would probably be >> languishing in oblivion. Everyone wants a faster Python so long as they >> don't have to pay for it, or do the work.) > > Hm, I didn't know the EU was funding PyPy. Okay. Cool. I don't know if they still receive funding, but I know that PyPy really only got going in a big way when they got a grant from the EU. I think it paid for at least one developer to work on it full time for a year. DuckDuckGo is probably your friend if you care about the details. -- Steve From ned at nedbatchelder.com Mon Feb 26 08:42:14 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 26 Feb 2018 08:42:14 -0500 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On 2/26/18 7:13 AM, bartc wrote: > On 26/02/2018 11:40, Chris Angelico wrote: >> On Mon, Feb 26, 2018 at 10:13 PM, bartc wrote: >>> Below is the first draft of a Python port of a program to do with >>> random >>> numbers. (Ported from my language, which in turned ported it from a C >>> program by George Marsaglia, the random number guy.) >>> >>> However, running it quickly exhausts the memory in my machine. The >>> reason is >>> that Python unhelpfully widens all results to bignums as needed. The >>> code >>> relies on calculations being modulo 2^64. >>> >>> Note that restricting integer ops to 64 bits probably still won't >>> work, as I >>> believe the numbers need to be unsigned. >> >> No, it's because the original implementation assumed integer >> wrap-around (at least, I think that's what's happening; I haven't >> analyzed the code in great detail). That means all your integer >> operations are doing two jobs: the one they claim to, and then a >> masking to 64 bits signed. That's two abstract operations that happen, >> due to the nature of the CPU, to work efficiently together. If you >> don't implement both halves of that in your Python port, you have >> failed at porting. What if you were porting a program from a 72-bit >> chip that assumed Binary Coded Decimal? Would you complain that C's >> integers are badly behaved? >> >> And that's without even asking whether a C program that assumes >> integer wrap-around counts as portable. At least with Python, you have >> a guarantee that integer operations are going to behave the same way >> on all compliant implementations of the language. > > > A C version is given below. (One I may have messed around with, which > I'm not sure works properly. For an original, google for Marsaglia and > KISS64 or SUPRKISS64.) > > Most integers are unsigned, which have well-defined overflow in C > (they just wrap as expected). In C, a mixed signed/unsigned op is > performed as unsigned. > > ----------------------------- > > /*?? SUPRKISS64.c, period 5*2^1320480*(2^64-1)?? */ > #include > #include > #include "timer.c" > ?static signed long long Q[20632], > ???????????????? carry=36243678541LL, > ???????????????? xcng=12367890123456LL, > ???????????????? xs=521288629546311LL, > ???????????????? indx=20632; > > ?#define CNG ( xcng=6906969069LL*xcng+123 ) > ?#define XS? ( xs^=xs<<13,xs^=xs>>17,xs^=xs<<43 ) > ?#define SUPR ( indx<20632 ? Q[indx++] : refill() ) > ?#define KISS SUPR+CNG+XS > > ?signed long long refill(void) { > ? int i; signed long long z,h; > ? for(i=0;i<20632;i++) { > ??? h = (carry&1); > ??? z = ((Q[i]<<41)>>1)+((Q[i]<<39)>>1)+(carry>>1); > ??? carry = (Q[i]>>23)+(Q[i]>>25)+(z>>63); > ??? Q[i] = ~((z<<1)+h); > ? } > ? indx=1; > ? return (Q[0]); > ?} > > ?int main() { > ? int i; signed long long x; > > ? for(i=0;i<20632;i++) Q[i]=CNG+XS; > > ? for(i=0;i<1000000000;i++) x=KISS; > > ? printf("Does x=4013566000157423768\n???? x=%llu.\n",x); > } > With proper 64-bit masking (def only64(x): return x & 0xFFFFFFFFFFFFFFFF), the Python version produces the correct answer using a reasonable amount of memory. Well, once you notice that the Python code had N=1e5, and the C code had N=1e9 :)?? If you want to experiment, with N=1e5, the final number should be 5255210926702073855. Also, I note that you said, "Most integers are unsigned", but the C code has them all declared as signed?? It doesn't seem to have mattered to your result, but I'm not an expert on C portability guarantees. --Ned. From rhodri at kynesim.co.uk Mon Feb 26 09:01:35 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 26 Feb 2018 14:01:35 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On 26/02/18 13:42, Ned Batchelder wrote: > Also, I note that you said, "Most integers are unsigned", but the C code > has them all declared as signed?? It doesn't seem to have mattered to > your result, but I'm not an expert on C portability guarantees. C explicitly leaves the behaviour of signed arithmetic overflow undefined, so you have no portability guarantees there. -- Rhodri James *-* Kynesim Ltd From bc at freeuk.com Mon Feb 26 09:04:53 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 14:04:53 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On 26/02/2018 13:42, Ned Batchelder wrote: > On 2/26/18 7:13 AM, bartc wrote: >> A C version is given below. (One I may have messed around with, which >> I'm not sure works properly. For an original, google for Marsaglia and >> KISS64 or SUPRKISS64.) >> >> Most integers are unsigned, which have well-defined overflow in C > With proper 64-bit masking (def only64(x): return x & > 0xFFFFFFFFFFFFFFFF), the Python version produces the correct answer > using a reasonable amount of memory. I did try sometime like that, but I must have missed something because I didn't get quite the same results as a working version. And with interpreted code, you tend not to test using loops of a billion iterations. Well, once you notice that the > Python code had N=1e5, and the C code had N=1e9 :)?? If you want to > experiment, with N=1e5, the final number should be 5255210926702073855. OK, I'll try that. > Also, I note that you said, "Most integers are unsigned", but the C code > has them all declared as signed?? It doesn't seem to have mattered to > your result, but I'm not an expert on C portability guarantees. The C code I first pasted used 'unsigned', but the main program logic wasn't right, and I found another version that looked better. That one used 'signed' for some reason, which I completely missed. Even if with C it works with either, the signed version might have 'undefined behaviour'. As said, google for the original; the ones I can see have 'unsigned'. But I can also see a Fortran version that just uses 'integer*8', which I believe is signed 64-bit. -- bartc From rosuav at gmail.com Mon Feb 26 09:34:51 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 01:34:51 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Tue, Feb 27, 2018 at 12:03 AM, Steven D'Aprano wrote: > On Mon, 26 Feb 2018 22:34:00 +1100, Chris Angelico wrote: >> Removing the GIL from CPython is not about "speeding up" the language or >> the interpreter, but about improving parallelism. > > It is about speeding up threaded code which is CPU-bound. I call that > speeding up the language :-) > > Hypothetically, it could also speed up even unthreaded code. In its purest sense, no, it cannot speed up unthreaded code. Since many Python programs are single-threaded, this restricts the benefits to those programs which can actually parallelize, but the penalties (mainly overhead) usually apply to all programs. > Some language features could internally launch threads and operate using > implicit parallelism. For instance, map(func, seq) could run in parallel > in separate threads whenever Python knew that func had no side-effects, > say for built-ins. There are also parallel algorithms for bignum > arithmetic. If threads were quick enough, I'm sure we could come up with > many more examples. Hypothetically speaking. These are ways of converting single-threaded code into multi-threaded code, which could then benefit from parallelization, which can reduce wall-time for the process as a whole. > (In practice, the old hope that parallel computing would speed everything > up turned out to be flawed. Relatively few tasks are embarrassingly > parallel, most have sequential bottlenecks, and many are inherently > sequential.) Exactly; and it goes further: many modern programmers do not think in terms of parallelism. Even event-driven code takes a bit of getting your head around. Why are async functions preferred over threads? It's not JUST because today's OSes have thread count limits far below socket limits - if that were the sole justification, we wouldn't have language-level support for async/await - it'd be a niche technique used only in the highest-throughput applications. No, async functions are preferred because they *reduce the points where context switching can occur*, thus making it easier *for the programmer*. If there were a way for the language to automatically run things in parallel, the biggest risk is that the programmer who wrote it can no longer understand it. >>> (If it weren't for the EU government funding it, PyPy would probably be >>> languishing in oblivion. Everyone wants a faster Python so long as they >>> don't have to pay for it, or do the work.) >> >> Hm, I didn't know the EU was funding PyPy. Okay. Cool. > > I don't know if they still receive funding, but I know that PyPy really > only got going in a big way when they got a grant from the EU. I think it > paid for at least one developer to work on it full time for a year. > DuckDuckGo is probably your friend if you care about the details. Meh, I don't care that much about what the EU does or doesn't fund. Was just a mild case of curiosity. I had about as much interest as my savings account accrued last year. I'm glad _someone_ funded PyPy, anyhow. It's a great demonstration of what can be done. ChrisA From steve+comp.lang.python at pearwood.info Mon Feb 26 09:41:36 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 26 Feb 2018 14:41:36 +0000 (UTC) Subject: Is there are good DRY fix for this painful design pattern? Message-ID: I have a class with a large number of parameters (about ten) assigned in `__init__`. The class then has a number of methods which accept *optional* arguments with the same names as the constructor/initialiser parameters. If those arguments are None, the defaults are taken from the instance attributes. An example might be something like this: class Foo: def __init__(self, bashful, doc, dopey, grumpy, happy, sleepy, sneezy): self.bashful = bashful # etc def spam(self, bashful=None, doc=None, dopey=None, grumpy=None, happy=None, sleepy=None, sneezy=None): if bashful is None: bashful = self.bashful if doc is None: doc = self.doc if dopey is None: dopey = self.dopey if grumpy is None: grumpy = self.grumpy if happy is None: happy = self.happy if sleepy is None: sleepy = self.sleepy if sneezy is None: sneezy = self.sneezy # now do the real work... def eggs(self, bashful=None, # etc... ): if bashful is None: bashful = self.bashful # and so on There's a lot of tedious boilerplate repetition in this, and to add insult to injury the class is still under active development with an unstable API, so every time I change one of the parameters, or add a new one, I have to change it in over a dozen places. Is there a good fix for this to reduce the amount of boilerplate? Thanks, -- Steve From bc at freeuk.com Mon Feb 26 09:45:33 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 14:45:33 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On 26/02/2018 14:34, Chris Angelico wrote: > I'm glad _someone_ funded PyPy, anyhow. It's a great demonstration of > what can be done. And it's good that /someone/ at least understands how PyPy works, as I don't think many people do. Apparently it doesn't optimise 'hot paths' within a Python program, but optimises hot paths in the special Python interpreter. One written in [R]Python. Or something... -- Bartc From antoon.pardon at vub.be Mon Feb 26 10:00:28 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Mon, 26 Feb 2018 16:00:28 +0100 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: On 26-02-18 15:41, Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. > > An example might be something like this: > > > class Foo: > def __init__(self, bashful, doc, dopey, grumpy, > happy, sleepy, sneezy): > self.bashful = bashful # etc > > def spam(self, bashful=None, doc=None, dopey=None, > grumpy=None, happy=None, sleepy=None, > sneezy=None): > if bashful is None: > bashful = self.bashful > if doc is None: > doc = self.doc > if dopey is None: > dopey = self.dopey > if grumpy is None: > grumpy = self.grumpy > if happy is None: > happy = self.happy > if sleepy is None: > sleepy = self.sleepy > if sneezy is None: > sneezy = self.sneezy > # now do the real work... Just throwing out an idea, but would it be possible to do what you want with a decorator? You let the wrapper do this kind of control and then pass control to the actual function with all values filled in? I'll try to work something out later. -- Antoon. From bc at freeuk.com Mon Feb 26 10:02:56 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 15:02:56 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On 26/02/2018 14:04, bartc wrote: > On 26/02/2018 13:42, Ned Batchelder wrote: > ?Well, once you notice that the >> Python code had N=1e5, and the C code had N=1e9 :)?? If you want to >> experiment, with N=1e5, the final number should be 5255210926702073855. > > OK, I'll try that. I have that Python version working now. It's necessary to apply that masking function to wherever numbers can get bigger. I don't know how long a 1-billion loop will take, but a 10-million loop took 46 seconds on Python 3.6, and 21 seconds on PyPy 2.7 from a couple of years ago. (And on Windows, which has a somewhat slower CPython than Linux.) Result should be x=11240129907685265998. By comparison, the C version compiled with -O3 took 0.11 seconds. (The C version I posted will work, if adjusted to a 10000000 loop, but you have to change 'signed' to 'unsigned'. Apparently they weren't interchangeable after all. I've no idea why I used 'signed' there. That version is rather cryptic, but it can be better written and without the macros, and it will run just as fast. (Marsaglia may have been hot with random number routines, but his C could have done with some work...) My interpreter, using 64-bit numbers, managed 4.8 seconds. But unsigned arithmetic, which is uncommon, is not accelerated.) --------------------------------------------------- Q=0 carry=36243678541 xcng=12367890123456 xs=521288629546311 indx=20632 def i64(x): return x & 0xFFFFFFFFFFFFFFFF def refill(): global Q, carry, indx for i in range(20632): h = carry & 1 z = i64(( i64((Q[i]<<41))>>1)+(i64((Q[i]<<39))>>1)+(carry>>1)) carry = i64((Q[i]>>23)+(Q[i]>>25)+(z>>63)) Q[i] = i64(~(i64(i64(z<<1)+h))) indx=1 return Q[0] def start(): global Q, carry, xcng, xs, indx Q=[0,]*20632 for i in range(20632): xcng=i64(6906969069 * xcng + 123) xs = i64(xs ^ (xs<<13)) xs = i64(xs ^ (xs>>17)) xs = i64(xs ^ (xs<<43)) Q[i] = i64(xcng + xs) N = 10000000 for i in range(N): if indx<20632: s = Q[indx] indx+=1 else: s = refill() xcng=i64(6906969069 * xcng + 123) xs = i64(xs ^ (xs<<13)) xs = i64(xs ^ (xs>>17)) xs = i64(xs ^ (xs<<43)) x = i64(s+xcng+xs) print ("Does x= 4013566000157423768") print (" x=",x) start() -- bartc From rosuav at gmail.com Mon Feb 26 10:06:07 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 02:06:07 +1100 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: On Tue, Feb 27, 2018 at 1:41 AM, Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. I have a very similar problem in a project of mine, and it's currently an unsolved problem. My current best theory is to basically just accept **kwargs for everything (so there's no None defaults), and then use chain_map for all lookups. Seems inefficient, but at least it'd be functional. Haven't implemented it though. If you get any better ideas, I'll be reading them too. ChrisA From vincent.vande.vyvre at telenet.be Mon Feb 26 10:09:08 2018 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Mon, 26 Feb 2018 16:09:08 +0100 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: Le 26/02/18 ? 15:41, Steven D'Aprano a ?crit?: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. > > An example might be something like this: > > > class Foo: > def __init__(self, bashful, doc, dopey, grumpy, > happy, sleepy, sneezy): > self.bashful = bashful # etc > > def spam(self, bashful=None, doc=None, dopey=None, > grumpy=None, happy=None, sleepy=None, > sneezy=None): > if bashful is None: > bashful = self.bashful > if doc is None: > doc = self.doc > if dopey is None: > dopey = self.dopey > if grumpy is None: > grumpy = self.grumpy > if happy is None: > happy = self.happy > if sleepy is None: > sleepy = self.sleepy > if sneezy is None: > sneezy = self.sneezy > # now do the real work... > > def eggs(self, bashful=None, # etc... > ): > if bashful is None: > bashful = self.bashful > # and so on > > > There's a lot of tedious boilerplate repetition in this, and to add > insult to injury the class is still under active development with an > unstable API, so every time I change one of the parameters, or add a new > one, I have to change it in over a dozen places. > > Is there a good fix for this to reduce the amount of boilerplate? > > > Thanks, > > > Maybe something like this: ??? def config(self, **kwargs): ??????? for key, value in kwargs.items(): ??????????? if value is not None: ??????????????? setattr(self, key, value) ??? def spam(self, **kwargs): ??????? self.config(kwargs) Vincent From rosuav at gmail.com Mon Feb 26 10:09:53 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 02:09:53 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Tue, Feb 27, 2018 at 2:02 AM, bartc wrote: > On 26/02/2018 14:04, bartc wrote: >> >> On 26/02/2018 13:42, Ned Batchelder wrote: > > >> Well, once you notice that the >>> >>> Python code had N=1e5, and the C code had N=1e9 :) If you want to >>> experiment, with N=1e5, the final number should be 5255210926702073855. >> >> >> OK, I'll try that. > > > I have that Python version working now. It's necessary to apply that masking > function to wherever numbers can get bigger. > > I don't know how long a 1-billion loop will take, but a 10-million loop took > 46 seconds on Python 3.6, and 21 seconds on PyPy 2.7 from a couple of years > ago. (And on Windows, which has a somewhat slower CPython than Linux.) You're still reimplementing the C code in Python, which is inefficient. Have you considered going back to the *actual algorithm* and implementing that idiomatically in Python? I think you'll find that (a) the code is more readable, and (b) the run time is much lower. ChrisA From antoon.pardon at vub.be Mon Feb 26 10:19:43 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Mon, 26 Feb 2018 16:19:43 +0100 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: On 26-02-18 15:41, Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. > This is what I came up with at short notice: def passthrough(func): def wrapper(self, **kwds): for argn in "bashful doc dopey grumpy happy sleepy sneezy".split(): if argn not in kwds: kwds[argn] = getattr(self, argn) func(self, **kwds) return wrapper class Foo: def __init__(self, bashful, doc, dopey, grumpy, happy, sleepy, sneezy): self.bashful = bashful self.doc = doc self.dopey = dopey self.grumpy = grumpy self.happy = happy self.sleepy = sleepy self.sneezy = sneezy @passthrough def spam(self, bashful=None, doc=None, dopey=None, grumpy=None, happy=None, sleepy=None, sneezy=None): print(bashful, doc, dopey, grumpy, happy, sleepy, sneezy) foo = Foo('a', 'b', 'c', 'd', 'e', 'f', 'g') foo.spam(happy = 1) # prints => a b c d 1 f g From ian.g.kelly at gmail.com Mon Feb 26 10:20:33 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 08:20:33 -0700 Subject: For Loop Dilema [python-list] In-Reply-To: References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Message-ID: On Sun, Feb 25, 2018 at 8:05 PM, INADA Naoki wrote: > https://docs.python.org/3.6/library/itertools.html#itertools.product I don't see how you would use itertools.product to do what the OP asked for. You could use itertools.chain.from_iterable, though: py> names = ['Jack', 'Susan'] py> list(chain.from_iterable(names)) ['J', 'a', 'c', 'k', 'S', 'u', 's', 'a', 'n'] From rhodri at kynesim.co.uk Mon Feb 26 10:28:54 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 26 Feb 2018 15:28:54 +0000 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: <4df77cd1-770d-90e4-0989-043ca9347b5a@kynesim.co.uk> On 26/02/18 14:41, Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. > > An example might be something like this: > > > class Foo: > def __init__(self, bashful, doc, dopey, grumpy, > happy, sleepy, sneezy): > self.bashful = bashful # etc > > def spam(self, bashful=None, doc=None, dopey=None, > grumpy=None, happy=None, sleepy=None, > sneezy=None): > if bashful is None: > bashful = self.bashful > if doc is None: > doc = self.doc > if dopey is None: > dopey = self.dopey > if grumpy is None: > grumpy = self.grumpy > if happy is None: > happy = self.happy > if sleepy is None: > sleepy = self.sleepy > if sneezy is None: > sneezy = self.sneezy > # now do the real work... > > def eggs(self, bashful=None, # etc... > ): > if bashful is None: > bashful = self.bashful > # and so on > > > There's a lot of tedious boilerplate repetition in this, and to add > insult to injury the class is still under active development with an > unstable API, so every time I change one of the parameters, or add a new > one, I have to change it in over a dozen places. > > Is there a good fix for this to reduce the amount of boilerplate? You could use dicts? Untested code: class Foo: def __init__(self, **kwds): self.defaults = kwds def spam(self, **kwds): vars = self.defaults.copy() vars.update(kwds) # Do the work with vars['bashful'] etc -- Rhodri James *-* Kynesim Ltd From rgaddi at highlandtechnology.invalid Mon Feb 26 11:30:35 2018 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 26 Feb 2018 08:30:35 -0800 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: <4df77cd1-770d-90e4-0989-043ca9347b5a@kynesim.co.uk> Message-ID: On 02/26/2018 07:28 AM, Rhodri James wrote: > On 26/02/18 14:41, Steven D'Aprano wrote: >> I have a class with a large number of parameters (about ten) assigned in >> `__init__`. The class then has a number of methods which accept >> *optional* arguments with the same names as the constructor/initialiser >> parameters. If those arguments are None, the defaults are taken from the >> instance attributes. >> >> An example might be something like this: >> >> >> class Foo: >> ???? def __init__(self, bashful, doc, dopey, grumpy, >> ??????????????????????? happy, sleepy, sneezy): >> ???????? self.bashful = bashful? # etc >> >> ???? def spam(self, bashful=None, doc=None, dopey=None, >> ??????????????????? grumpy=None, happy=None, sleepy=None, >> ??????????????????? sneezy=None): >> ???????? if bashful is None: >> ???????????? bashful = self.bashful >> ???????? if doc is None: >> ???????????? doc = self.doc >> ???????? if dopey is None: >> ???????????? dopey = self.dopey >> ???????? if grumpy is None: >> ???????????? grumpy = self.grumpy >> ???????? if happy is None: >> ???????????? happy = self.happy >> ???????? if sleepy is None: >> ???????????? sleepy = self.sleepy >> ???????? if sneezy is None: >> ???????????? sneezy = self.sneezy >> ???????? # now do the real work... >> >> ???? def eggs(self, bashful=None, # etc... >> ??????????????????? ): >> ???????? if bashful is None: >> ???????????? bashful = self.bashful >> ???????? # and so on >> >> There's a lot of tedious boilerplate repetition in this, and to add >> insult to injury the class is still under active development with an >> unstable API, so every time I change one of the parameters, or add a new >> one, I have to change it in over a dozen places. >> >> Is there a good fix for this to reduce the amount of boilerplate? > > You could use dicts?? Untested code: > > class Foo: > ??? def __init__(self, **kwds): > ??????? self.defaults = kwds > > ??? def spam(self, **kwds): > ??????? vars = self.defaults.copy() > ??????? vars.update(kwds) > ????# Do the work with vars['bashful'] etc > > That's what I was about to suggest (and equally fail to test). It at least segregates the set of "instance variables I want to have auto-default behavior" from the rest of them. Yeah, **kwargs is clumsy, but it gets the job done. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From bc at freeuk.com Mon Feb 26 11:31:48 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 16:31:48 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On 26/02/2018 15:09, Chris Angelico wrote: > On Tue, Feb 27, 2018 at 2:02 AM, bartc wrote: >> On 26/02/2018 14:04, bartc wrote: >>> >>> On 26/02/2018 13:42, Ned Batchelder wrote: >> >> >>> Well, once you notice that the >>>> >>>> Python code had N=1e5, and the C code had N=1e9 :) If you want to >>>> experiment, with N=1e5, the final number should be 5255210926702073855. >>> >>> >>> OK, I'll try that. >> >> >> I have that Python version working now. It's necessary to apply that masking >> function to wherever numbers can get bigger. >> >> I don't know how long a 1-billion loop will take, but a 10-million loop took >> 46 seconds on Python 3.6, and 21 seconds on PyPy 2.7 from a couple of years >> ago. (And on Windows, which has a somewhat slower CPython than Linux.) > > You're still reimplementing the C code in Python, which is > inefficient. Have you considered going back to the *actual algorithm* > and implementing that idiomatically in Python? I think you'll find > that (a) the code is more readable, and (b) the run time is much > lower. Do you really think that? The algorithm seems to require this sequence of calculations to be gone through. Otherwise anything more efficient can also be done in any language. So how would idiomatic Python help, by seeing if such a routine already exists in some library? That wouldn't be playing the game. If it helps, I remember playing with a version in Algol 68 Genie (interpreted). Still buggy as the output is different, it does about the same amount of work. A 10-million loop would take an estimated 1000 seconds, on the same machine that CPython took 46 seconds. So four magnitudes slower than C. -- bartc From ned at nedbatchelder.com Mon Feb 26 11:32:40 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 26 Feb 2018 11:32:40 -0500 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: <8ad784c2-bd6c-5714-61d2-6f22f0900324@nedbatchelder.com> On 2/26/18 10:09 AM, Chris Angelico wrote: > On Tue, Feb 27, 2018 at 2:02 AM, bartc wrote: >> On 26/02/2018 14:04, bartc wrote: >>> On 26/02/2018 13:42, Ned Batchelder wrote: >> >>> Well, once you notice that the >>>> Python code had N=1e5, and the C code had N=1e9 :) If you want to >>>> experiment, with N=1e5, the final number should be 5255210926702073855. >>> >>> OK, I'll try that. >> >> I have that Python version working now. It's necessary to apply that masking >> function to wherever numbers can get bigger. >> >> I don't know how long a 1-billion loop will take, but a 10-million loop took >> 46 seconds on Python 3.6, and 21 seconds on PyPy 2.7 from a couple of years >> ago. (And on Windows, which has a somewhat slower CPython than Linux.) > You're still reimplementing the C code in Python, which is > inefficient. Have you considered going back to the *actual algorithm* > and implementing that idiomatically in Python? I think you'll find > that (a) the code is more readable, and (b) the run time is much > lower. > > Don't bother: C will win this kind of race every time. --Ned. From __peter__ at web.de Mon Feb 26 11:39:43 2018 From: __peter__ at web.de (Peter Otten) Date: Mon, 26 Feb 2018 17:39:43 +0100 Subject: Is there are good DRY fix for this painful design pattern? References: Message-ID: Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. > > An example might be something like this: > > > class Foo: > def __init__(self, bashful, doc, dopey, grumpy, > happy, sleepy, sneezy): > self.bashful = bashful # etc > > def spam(self, bashful=None, doc=None, dopey=None, > grumpy=None, happy=None, sleepy=None, > sneezy=None): > if bashful is None: > bashful = self.bashful > if doc is None: > doc = self.doc > if dopey is None: > dopey = self.dopey > if grumpy is None: > grumpy = self.grumpy > if happy is None: > happy = self.happy > if sleepy is None: > sleepy = self.sleepy > if sneezy is None: > sneezy = self.sneezy > # now do the real work... > > def eggs(self, bashful=None, # etc... > ): > if bashful is None: > bashful = self.bashful > # and so on > > > There's a lot of tedious boilerplate repetition in this, and to add > insult to injury the class is still under active development with an > unstable API, so every time I change one of the parameters, or add a new > one, I have to change it in over a dozen places. > > Is there a good fix for this to reduce the amount of boilerplate? I have not yet looked into dataclasses. Don't they handle the __init__() part? Anyway, here's my attempt to make spam() less spammy: $ cat attrs_to_args_decorator.py import functools import inspect def add_defaults(f): argnames = inspect.getfullargspec(f).args[1:] @functools.wraps(f) def wrapper(self, *args, **kw): args = [ getattr(self, name) if value is None else value for name, value in zip(argnames, args) ] for name in argnames[len(args):]: if name not in kw or kw[name] is None: kw[name] = getattr(self, name) return f(self, *args, **kw) return wrapper def update_attrs(kw): self = kw.pop("self") for name, value in kw.items(): setattr(self, name, value) class Foo: def __init__(self, bashful, doc, dopey, grumpy, happy, sleepy, sneezy): update_attrs(locals()) @add_defaults def spam(self, bashful=None, doc=None, dopey=None, grumpy=None, happy=None, sleepy=None, sneezy=None): return "{}-{}-{}".format(bashful, doc, sneezy) def __repr__(self): return "Foo({})".format( ", ".join( "{}={!r}".format(*pair) for pair in sorted(self.__dict__.items()) ) ) if __name__ == "__main__": foo = Foo("bashful", "doc", "dopey", "grumpy", "happy", "sleepy", "sneezy") print(foo) print(foo.spam()) print(foo.spam(bashful="BASHFUL")) print(foo.spam(bashful="BASHFUL", doc="DOC")) print(foo.spam("BASHFUL")) print(foo.spam("BASHFUL", "DOC")) print(foo.spam("BASHFUL", "DOC", sneezy="SNEEZY")) $ python3 attrs_to_args_decorator.py Foo(bashful='bashful', doc='doc', dopey='dopey', grumpy='grumpy', happy='happy', sleepy='sleepy', sneezy='sneezy') bashful-doc-sneezy BASHFUL-doc-sneezy BASHFUL-DOC-sneezy BASHFUL-doc-sneezy BASHFUL-DOC-sneezy BASHFUL-DOC-SNEEZY $ From spluque at gmail.com Mon Feb 26 11:53:54 2018 From: spluque at gmail.com (Seb) Date: Mon, 26 Feb 2018 10:53:54 -0600 Subject: matrix multiplication References: <87lgfhnesw.fsf@otaria.sebmel.org> Message-ID: <87efl7pu8t.fsf@gmail.com> On Sun, 25 Feb 2018 18:52:14 -0500, Terry Reedy wrote: [...] > numpy has a matrix multiply function and now the '@' matrix multiply > operator. Yes, but what I was wondering is whether there's a faster way of multiplying each row (1x3) of a matrix by another matrix (3x3), compared to looping through the matrix row by row as shown in the code. Perhaps I'm not understanding how to use the broadcasting features of `matmul`. -- Seb From ben.usenet at bsb.me.uk Mon Feb 26 12:05:09 2018 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Mon, 26 Feb 2018 17:05:09 +0000 Subject: How to make Python run as fast (or faster) than Julia References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: <87lgffsmuy.fsf@bsb.me.uk> bartc writes: > A C version is given below. (One I may have messed around with, which > I'm not sure works properly. For an original, google for Marsaglia and > KISS64 or SUPRKISS64.) The version I know uses unsigned integers. Did you change then to signed? For a Python version, go back to the original C and work from there. -- Ben. From ian.g.kelly at gmail.com Mon Feb 26 12:46:02 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 10:46:02 -0700 Subject: matrix multiplication In-Reply-To: <87efl7pu8t.fsf@gmail.com> References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> Message-ID: On Mon, Feb 26, 2018 at 9:53 AM, Seb wrote: > On Sun, 25 Feb 2018 18:52:14 -0500, > Terry Reedy wrote: > > [...] > >> numpy has a matrix multiply function and now the '@' matrix multiply >> operator. > > Yes, but what I was wondering is whether there's a faster way of > multiplying each row (1x3) of a matrix by another matrix (3x3), compared > to looping through the matrix row by row as shown in the code. Perhaps > I'm not understanding how to use the broadcasting features of `matmul`. >From the matmul documentation: "If either argument is N-D, N > 2, it is treated as a stack of matrices residing in the last two indexes and broadcast accordingly." So you probably want your uvw array to be 1000x1x3 instead of 1000x3. Then the result of multiplying them should be 1000 1x3 matrices. From rantingrickjohnson at gmail.com Mon Feb 26 13:14:14 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 26 Feb 2018 10:14:14 -0800 (PST) Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Monday, February 26, 2018 at 3:59:40 AM UTC-6, Steven D'Aprano wrote: > On Sun, 25 Feb 2018 20:22:17 -0800, Rick Johnson wrote: > (We tried painting Go Faster stripes on the server, and it > didn't work.) Well of course the server won't work after you drip water- based paint all over the circuit boards. Way to go! In the future i would suggest draping some old linens over the electrical bits before throwing your paint around (just be sure to consult with the wife before rummaging through her linen closet, as men have been known to meet an untimely demise for doing much less) . And might i suggest some skulls and faux flames? Hey, it may not make the thing go any faster, but it will surely make the other kids jealous. There's no reason why apple and microsoft should be allowed to suck up every last merchandising dollar. > There's no point suggesting such major changes to Python > that requires going back to the drawing board, to Python > 0.1 or earlier, and changing the entire execution and > memory model of the language. Absolutely not. Probably too late for that now. I'm only suggesting that we, when at all possible, provide a switch to disable these lazy-features in those rare cases when they do create bottlenecks. Lazy iterators and dict-views are great ideas, but i'm sure there are instances which prove them to be PITA and even a bottleneck. From arya.kumar2494 at gmail.com Mon Feb 26 13:26:55 2018 From: arya.kumar2494 at gmail.com (arya.kumar2494 at gmail.com) Date: Mon, 26 Feb 2018 10:26:55 -0800 (PST) Subject: For Loop Dilema [python-list] In-Reply-To: References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Message-ID: <4334b6b2-4624-4cb5-aef3-267c58a04b30@googlegroups.com> On Monday, February 26, 2018 at 8:51:35 PM UTC+5:30, Ian wrote: > On Sun, Feb 25, 2018 at 8:05 PM, INADA Naoki wrote: > > https://docs.python.org/3.6/library/itertools.html#itertools.product > > I don't see how you would use itertools.product to do what the OP > asked for. You could use itertools.chain.from_iterable, though: > > py> names = ['Jack', 'Susan'] > py> list(chain.from_iterable(names)) > ['J', 'a', 'c', 'k', 'S', 'u', 's', 'a', 'n'] if you want to access a dict per say insisde a list or maybe some data i.e deep nested in some mixture of ds wont it be relevant to use a single for statement to query the data out.(Just a thought) Students = [{"name":John,"age":16},{"name":Maria,"age":18}] for info in student in Students: print(student[info]) From arya.kumar2494 at gmail.com Mon Feb 26 13:27:23 2018 From: arya.kumar2494 at gmail.com (arya.kumar2494 at gmail.com) Date: Mon, 26 Feb 2018 10:27:23 -0800 (PST) Subject: For Loop Dilema [python-list] In-Reply-To: References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Message-ID: On Monday, February 26, 2018 at 6:20:06 AM UTC+5:30, Ian wrote: > On Sun, Feb 25, 2018 at 11:19 AM, wrote: > > Why we don?t use: > > > > for _ in _ in _ > > > > Instead of > > > > for _ in _: > > for _ in _: > > > > Ex: > > > > Names = ["Arya","Pupun"] > > > > for name in Names: > > for c in name: > > print(c) > > > > instead use: > > > > for c in name in Names: > > print(c) > > It doesn't seem very intuitive (doesn't follow proper English > phrasing, for instance) and I don't think it's a common enough > situation to warrant adding a special syntax for it. But if you really > want it, you could use something like this: > > def double_for(iterable): > for outer in iterable: > yield from outer > > for c in double_for(Names): > print(c) > > But I don't think this is any clearer than making the loops explicit. Thank you. From arya.kumar2494 at gmail.com Mon Feb 26 13:27:42 2018 From: arya.kumar2494 at gmail.com (arya.kumar2494 at gmail.com) Date: Mon, 26 Feb 2018 10:27:42 -0800 (PST) Subject: For Loop Dilema [python-list] In-Reply-To: References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> Message-ID: <99613d74-ddfc-4472-b16c-abbe6d9b5e0c@googlegroups.com> On Monday, February 26, 2018 at 12:57:25 PM UTC+5:30, Chris Angelico wrote: > On Mon, Feb 26, 2018 at 5:19 AM, wrote: > > Why we don?t use: > > > > for _ in _ in _ > > > > Instead of > > > > for _ in _: > > for _ in _: > > > > Ex: > > > > Names = ["Arya","Pupun"] > > > > for name in Names: > > for c in name: > > print(c) > > > > instead use: > > > > for c in name in Names: > > print(c) > > Because programming is all about building up a program from > primitives. The number of times when we need this kind of nested loop > (with absolutely nothing in between the loops) is way too small to > justify dedicated syntax. > > ChrisA Thank you. From arya.kumar2494 at gmail.com Mon Feb 26 13:28:22 2018 From: arya.kumar2494 at gmail.com (arya.kumar2494 at gmail.com) Date: Mon, 26 Feb 2018 10:28:22 -0800 (PST) Subject: For Loop Dilema [python-list] In-Reply-To: <7690174c-8a58-4bdd-accd-4410508f4cc1@googlegroups.com> References: <57e2f9cf-1447-4206-9220-1aae8173879f@googlegroups.com> <7690174c-8a58-4bdd-accd-4410508f4cc1@googlegroups.com> Message-ID: <19f2949e-0e55-4ec7-af0a-39855c2335c5@googlegroups.com> On Monday, February 26, 2018 at 6:23:24 AM UTC+5:30, Rick Johnson wrote: > On Sunday, February 25, 2018 at 12:19:56 PM UTC-6, arya.ku... at gmail.com wrote: > > > Ex: > > > > Names = ["Arya","Pupun"] > > > > for name in Names: > > for c in name: > > print(c) > > > > instead use: > > > > for c in name in Names: > > print(c) > > Hmm. Why stop there? > > bit = ["kibbles"] > bits = [bit, bit] > bitts = [bits, bits] > for kibbles in bit in bits in bitts: > do_something(kibbles) My thought exactly.. :) From ian.g.kelly at gmail.com Mon Feb 26 13:31:18 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 11:31:18 -0700 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: On Mon, Feb 26, 2018 at 8:06 AM, Chris Angelico wrote: > On Tue, Feb 27, 2018 at 1:41 AM, Steven D'Aprano > wrote: >> I have a class with a large number of parameters (about ten) assigned in >> `__init__`. The class then has a number of methods which accept >> *optional* arguments with the same names as the constructor/initialiser >> parameters. If those arguments are None, the defaults are taken from the >> instance attributes. > > I have a very similar problem in a project of mine, and it's currently > an unsolved problem. My current best theory is to basically just > accept **kwargs for everything (so there's no None defaults), and then > use chain_map for all lookups. Seems inefficient, but at least it'd be > functional. Haven't implemented it though. > > If you get any better ideas, I'll be reading them too. I would be tempted to take this and then formalize it into a Context class that would be passed to the various methods and initializers rather than passing all the sundry arguments directly. Similar to what the decimal class uses although perhaps without the idea of a global context (which just leads to trouble down the road). From bc at freeuk.com Mon Feb 26 13:52:54 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 18:52:54 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <87lgffsmuy.fsf@bsb.me.uk> References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> <87lgffsmuy.fsf@bsb.me.uk> Message-ID: On 26/02/2018 17:05, Ben Bacarisse wrote: > bartc writes: > >> A C version is given below. (One I may have messed around with, which >> I'm not sure works properly. For an original, google for Marsaglia and >> KISS64 or SUPRKISS64.) > > The version I know uses unsigned integers. Did you change then to signed? Yeah, I don't know what I was doing with that version. > For a Python version, go back to the original C and work from there. The original C makes confusing use of macros and comma operators. A version without macros or comma expressions is here (tweaked from generated C): https://pastebin.com/raw/k4jFK5TN This runs the 1-billion loop in 6 seconds. -- bartc From kirillbalunov at gmail.com Mon Feb 26 13:55:35 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Mon, 26 Feb 2018 21:55:35 +0300 Subject: Questions about `locals` builtin Message-ID: Hi, I am a little bit confused with `locals` builtin in these moments: 1. The documentation says that _free varaibles_ are returned, which seems incorrect description. In my mind the term free variable refers to variables used in a function that are not local variables nor parameters of that function. In docs: "Update and return a dictionary representing the current local symbol table. Free variables are returned by `locals()` when it is called in function blocks, but not in class blocks." >>> def func(): b = 12 print(locals(), ' : ', id(locals)) c = 13 print(locals(), ' : ', id(locals)) >>> print("Free variables: ", func.__code__.co_names) >>> print("Local variables: ", func.__code__.co_varnames) Free variables: ('print', 'locals', 'id') Local variables: ('b', 'c') 2. The documentation has a note that "The contents of this dictionary should not be modified". Which implies that it is a read only mapping. So the question why it is `dict` instead of `types.MappingProxyType`? 3. There is one more moment: local variables had been determined when function was compiled. But `locals` returns _some_ current runtime copy. I find this also confusing: >>> def func1(): loc = locals() b = 12 return loc >>> def func2(): b = 12 loc = locals() return loc >>> func1() { } >>> func2() {'b': 12} With kind regards, -gdg From rantingrickjohnson at gmail.com Mon Feb 26 14:37:21 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 26 Feb 2018 11:37:21 -0800 (PST) Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <700aebe6-0d27-43c0-a05c-f64ad75e50f3@googlegroups.com> Message-ID: On Monday, February 26, 2018 at 4:39:22 AM UTC-6, Steven D'Aprano wrote: > On Sun, 25 Feb 2018 19:26:12 -0800, Rick Johnson wrote: > > > On Friday, February 23, 2018 at 8:48:55 PM UTC-6, Steven D'Aprano wrote: > > [...] > > > Take the Fibonacci double-recursion benchmark. Okay, it > > > tests how well your language does at making millions of > > > function calls. Why? > > > > Because making "millons of function calls" is what > > software *DOES*! > > You're right -- I worded that badly, and ended up saying > something I didn't really mean. Mea culpa. Of course over > the course of the entire program run, most non-trivial > programmers will end up having millions (if not billions) > of function calls *in total*. But they're not all happening > *together*. So what? Latency is latency. And whether it occurs over the course of one heavily recursive algorithm that constitutes the depth and breadth of an entire program (a la fib()), or it is the incremental cumulative consequence of the entire program execution, the fact remains that function call overhead contributes to a significant portion of the latency inherent in some trivial, and *ALL* non-trivial, modern software. > > What I tried to get across is, how often do we use millions > of function calls to get one value? All the time! For, in the abstract, the fundamental function of software is to extract a "value" from one or more inputs -- is it not? So, whether that "value" be: (1) a document in an office productivity suite, or (2) a flight path for a mars lander or a jumbo jet or child's quad-copter, or (3) a modified image in a image processing program, or (4) a design drawing or technical schematic for a mars rover bot, or (5) an HD quality streaming porno flick, or yes, (6) even a recursive function which unwittingly underscores some of the less attractive aspects of the Python language (aka: function call overhead!) -- all of these examples, and countless others, extract a value from inputs. Of course, much like physical traits i suppose, some "values" are more desirable than others. > > By default, Python will only allow one thousand function > calls in a stack before raising RecursionError. By _default_, yes. > > py> def bad(): > ... return bad() > ... > py> bad() > Traceback (most recent call last): > File "", line 1, in > File "", line 2, in bad > [ previous line repeats 998 times ] > RecursionError: maximum recursion depth exceeded > > > There's nothing unique about recursion that will trigger > this error. Any call stack of more than 1000 function calls > will do it. So forget about code getting a million > functions deep. The average is probably more like a dozen, > perhaps a hundred tops. I'm very glad you brought this up, because it proves my main point that, Python's maximum recursion error is one feature they got _right_. If you don't like the default recursion limit of 1000, or maybe 1000 recursions is breaking a valid recursive algorithm, you can set the recursion limit to whatever you like... ## Python 2 ## >>> import sys >>> sys.setrecursionlimit >>> sys.setrecursionlimit(10) >>> count = 1 >>> def bad(): ... global count ... print count ... count += 1 ... bad() ... >>> bad() 1 2 3 4 5 6 7 8 9 Traceback (most recent call last): File "", line 1, in File "", line 5, in bad File "", line 5, in bad File "", line 5, in bad File "", line 5, in bad File "", line 5, in bad File "", line 5, in bad File "", line 5, in bad File "", line 5, in bad File "", line 5, in bad RuntimeError: maximum recursion depth exceeded It is however unfortunate that we get slamed with N recursive exception messages :-(, but hey, at least we can modify the maximum recursion limit! It's a step in the right direction. :-) > [...] > > > For most application code, executing the function is far > > > more costly than the overhead of calling it, > > > > What? Your functions are only called _once_ during the > > lifetime of your code execution? Enough with the leg > > pulling. > > Let me explain what I intended to say. Here is a really > poor estimate of the overhead of calling a function on my > computer, measured in iPython. > > [...snip valid examples...] > > Now I know that this is not always the case. But I maintain > that for *most* (but not all) practical, real-world code > that actually does something useful, the overhead of > calling the functions is *usually* (but not always) low > compared to the cost of actually doing whatever it is that > the functions do. True. But again you failed to address my main point that: "Lazy features (when feasible) must be offered _optionally_". Yes, function call overhead may be tiny in relation to the code encapsulated within (at least for non-trivial code), but the fact remains that function calls are inherent in modern software, and as a result, there is a compelling argument to be made that function calls should therefore be as efficient as possible. In reality, one cannot write modern software without using lots of functions and calling (at least) some subset of those functions *MANY* times during the execution of the program. And we're not just talking about functions who's parent node is module scope. Nope. Methods are functions too, albeit, a special bound class of functions. And the Python std-lib is chalk full of functions, many of which are utilized every second of every day to do lots of heavy lifting. So in conclusion, if anyone out there is not writing tons of functions to solve their problems (or at least calling tons of library functions to solve their problems), well then, their software is either really *REALLY* simplistic, or they are doing something really *REALLY* wrong. And from from that conclusion we now come face to face with the reality that function call efficiency may be the most important optimization that Python could ever make. From kirillbalunov at gmail.com Mon Feb 26 14:38:39 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Mon, 26 Feb 2018 22:38:39 +0300 Subject: Implicit conversion to str in str.join method? Message-ID: Currently `str.join` raises `TypeError` if there are any non-string values in iterable, including `bytes` objects. Is it an awful idea to implicitly _cast_ elements in iterable to their `str` or `repr` representation? Was this question adressed before? As for me there are two valid points: On one hand "Explicit is beter than implicit" and on the other "Practicality beats purity". May be I'm the only one who finds that it is rather boring to write somehting like: ", ".join(str(i) for i in iterable) when iterable contains non-string values, instead of: ", ".join(iterable) I don't know how much overhead will yield for the case when iterable contains only strings...and if it is possible to optimize this case. This change is backward compatible, but as I wrote above, it can be perceived badly and yield a huge overhead for general case that it is not even worth discussing. What do you think? With kind regards, -gdg From rosuav at gmail.com Mon Feb 26 14:43:04 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 06:43:04 +1100 Subject: Implicit conversion to str in str.join method? In-Reply-To: References: Message-ID: On Tue, Feb 27, 2018 at 6:38 AM, Kirill Balunov wrote: > Currently `str.join` raises `TypeError` if there are any non-string values > in iterable, including `bytes` objects. Is it an awful idea to implicitly > _cast_ elements in iterable to their `str` or `repr` representation? Was > this question adressed before? > > As for me there are two valid points: On one hand "Explicit is beter than > implicit" and on the other "Practicality beats purity". May be I'm the only > one who finds that it is rather boring to write somehting like: > > ", ".join(str(i) for i in iterable) > > when iterable contains non-string values, instead of: > > ", ".join(iterable) > > I don't know how much overhead will yield for the case when iterable > contains only strings...and if it is possible to optimize this case. This > change is backward compatible, but as I wrote above, it can be perceived > badly and yield a huge overhead for general case that it is not even worth > discussing. What do you think? This would be a perfect job for map. ", ".join(map(str, iterable)) If this is for display, you could also just print the values directly: print(*iterable, sep=", ") ChrisA From rosuav at gmail.com Mon Feb 26 14:50:21 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 06:50:21 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <700aebe6-0d27-43c0-a05c-f64ad75e50f3@googlegroups.com> Message-ID: On Tue, Feb 27, 2018 at 6:37 AM, Rick Johnson wrote: > On Monday, February 26, 2018 at 4:39:22 AM UTC-6, Steven D'Aprano wrote: >> On Sun, 25 Feb 2018 19:26:12 -0800, Rick Johnson wrote: >> >> > On Friday, February 23, 2018 at 8:48:55 PM UTC-6, Steven D'Aprano wrote: >> > [...] >> > > Take the Fibonacci double-recursion benchmark. Okay, it >> > > tests how well your language does at making millions of >> > > function calls. Why? >> > >> > Because making "millons of function calls" is what >> > software *DOES*! >> >> You're right -- I worded that badly, and ended up saying >> something I didn't really mean. Mea culpa. Of course over >> the course of the entire program run, most non-trivial >> programmers will end up having millions (if not billions) >> of function calls *in total*. But they're not all happening >> *together*. > > So what? Latency is latency. And whether it occurs over the > course of one heavily recursive algorithm that constitutes > the depth and breadth of an entire program (a la fib()), or > it is the incremental cumulative consequence of the entire > program execution, the fact remains that function call > overhead contributes to a significant portion of the latency > inherent in some trivial, and *ALL* non-trivial, modern > software. By saying "the fact remains", you're handwaving away the work of actually measuring that function call overhead is "significant". Can you show me those numbers? Steve's point is that it is NOT significant, because non-toy functions have non-trivial bodies. If you wish to disagree, you have to demonstrate that the function call is *itself* costly, even when there's a real body to it. > I'm very glad you brought this up, because it proves my main > point that, Python's maximum recursion error is one feature > they got _right_. If you don't like the default recursion > limit of 1000, or maybe 1000 recursions is breaking a valid > recursive algorithm, you can set the recursion limit to > whatever you like... > > ## Python 2 ## > >>> import sys > >>> sys.setrecursionlimit > > >>> sys.setrecursionlimit(10) > >>> count = 1 > >>> def bad(): > ... global count > ... print count > ... count += 1 > ... bad() > ... > >>> bad() > 1 > 2 > 3 > 4 > 5 > 6 > 7 > 8 > 9 > Traceback (most recent call last): > File "", line 1, in > File "", line 5, in bad > File "", line 5, in bad > File "", line 5, in bad > File "", line 5, in bad > File "", line 5, in bad > File "", line 5, in bad > File "", line 5, in bad > File "", line 5, in bad > File "", line 5, in bad > RuntimeError: maximum recursion depth exceeded > > It is however unfortunate that we get slamed with N > recursive exception messages :-(, but hey, at least we can > modify the maximum recursion limit! It's a step in the right > direction. :-) Ah, been a while since you tried Python 3, is it? :) [Previous line repeated X more times] ChrisA From kirillbalunov at gmail.com Mon Feb 26 14:54:26 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Mon, 26 Feb 2018 22:54:26 +0300 Subject: Implicit conversion to str in str.join method? In-Reply-To: References: Message-ID: > > print(*iterable, sep=", ") Thanks, I apologize :-) and why I always manage to find complicated ways... With kind regards, -gdg 2018-02-26 22:43 GMT+03:00 Chris Angelico : > On Tue, Feb 27, 2018 at 6:38 AM, Kirill Balunov > wrote: > > Currently `str.join` raises `TypeError` if there are any non-string > values > > in iterable, including `bytes` objects. Is it an awful idea to implicitly > > _cast_ elements in iterable to their `str` or `repr` representation? Was > > this question adressed before? > > > > As for me there are two valid points: On one hand "Explicit is beter than > > implicit" and on the other "Practicality beats purity". May be I'm the > only > > one who finds that it is rather boring to write somehting like: > > > > ", ".join(str(i) for i in iterable) > > > > when iterable contains non-string values, instead of: > > > > ", ".join(iterable) > > > > I don't know how much overhead will yield for the case when iterable > > contains only strings...and if it is possible to optimize this case. This > > change is backward compatible, but as I wrote above, it can be perceived > > badly and yield a huge overhead for general case that it is not even > worth > > discussing. What do you think? > > This would be a perfect job for map. > > ", ".join(map(str, iterable)) > > If this is for display, you could also just print the values directly: > > print(*iterable, sep=", ") > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Mon Feb 26 14:58:37 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 06:58:37 +1100 Subject: Implicit conversion to str in str.join method? In-Reply-To: References: Message-ID: On Tue, Feb 27, 2018 at 6:54 AM, Kirill Balunov wrote: >> print(*iterable, sep=", ") > > > Thanks, I apologize :-) and why I always manage to find complicated ways... Hey, that's why we have the list :) I call this a success. ChrisA From grant.b.edwards at gmail.com Mon Feb 26 15:03:55 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 26 Feb 2018 20:03:55 +0000 (UTC) Subject: Implicit conversion to str in str.join method? References: Message-ID: On 2018-02-26, Kirill Balunov wrote: > Currently `str.join` raises `TypeError` if there are any non-string values > in iterable, including `bytes` objects. Is it an awful idea to implicitly > _cast_ elements in iterable to their `str` or `repr` representation? Yes, that's an awful idea. PHP pulls stunts like that behind your back, and it's a huge source of bugs. -- Grant Edwards grant.b.edwards Yow! Let's all show human at CONCERN for REVERAND MOON's gmail.com legal difficulties!! From marco.nawijn at colosso.nl Mon Feb 26 15:09:32 2018 From: marco.nawijn at colosso.nl (marco.nawijn at colosso.nl) Date: Mon, 26 Feb 2018 12:09:32 -0800 (PST) Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: On Monday, February 26, 2018 at 3:44:14 PM UTC+1, Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. > > An example might be something like this: > > > class Foo: > def __init__(self, bashful, doc, dopey, grumpy, > happy, sleepy, sneezy): > self.bashful = bashful # etc > > def spam(self, bashful=None, doc=None, dopey=None, > grumpy=None, happy=None, sleepy=None, > sneezy=None): > if bashful is None: > bashful = self.bashful > if doc is None: > doc = self.doc > if dopey is None: > dopey = self.dopey > if grumpy is None: > grumpy = self.grumpy > if happy is None: > happy = self.happy > if sleepy is None: > sleepy = self.sleepy > if sneezy is None: > sneezy = self.sneezy > # now do the real work... > > def eggs(self, bashful=None, # etc... > ): > if bashful is None: > bashful = self.bashful > # and so on > > > There's a lot of tedious boilerplate repetition in this, and to add > insult to injury the class is still under active development with an > unstable API, so every time I change one of the parameters, or add a new > one, I have to change it in over a dozen places. > > Is there a good fix for this to reduce the amount of boilerplate? > > > Thanks, > > > > -- > Steve What about something along these lines: from inspect import getargspec, getargvalues class Demo(object): def __init__(self, a=None, b=10, c='oops'): spec = getargspec(self.__init__) for key, value in zip(spec.args[1:], spec.defaults): setattr(self, key, value) def foo(self, *args, **kwargs): assert len(args) == 0 for k, v in kwargs.items(): try: getattr(self, k) except AttributeError: print('*** error: attribute {} does not exist.'.format(k)) setattr(self, k, v) d = Demo() print(d.a) d.foo(a=3.14) print(d.a) d.foo(q='this will raise') So, use the inspect module to detect the valid arguments from the class initializer. Then use **kwargs in every class method. It would be nice if the full method signature can be kept, but I have not yet figured out if this is possible. Marco From bc at freeuk.com Mon Feb 26 15:27:36 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 20:27:36 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <700aebe6-0d27-43c0-a05c-f64ad75e50f3@googlegroups.com> Message-ID: On 26/02/2018 19:50, Chris Angelico wrote: > On Tue, Feb 27, 2018 at 6:37 AM, Rick Johnson >> So what? Latency is latency. And whether it occurs over the >> course of one heavily recursive algorithm that constitutes >> the depth and breadth of an entire program (a la fib()), or >> it is the incremental cumulative consequence of the entire >> program execution, the fact remains that function call >> overhead contributes to a significant portion of the latency >> inherent in some trivial, and *ALL* non-trivial, modern >> software. Take the last bit of Python I posted, which was that RNG test. It uses this function: def i64(x): return x & 0xFFFFFFFFFFFFFFFF This is a small function, but it can't be a toy one as it was suggested by Ned Batchelder. And the test program wasn't at all recursive. Running the program with N=10_000_000 took 46 seconds. Replacing the i64() call with '&m' (where m is a global set to 0xFFFFFFFFFFFFFFFF), it took 38 seconds. So putting that fix into a function, which is convenient for coding, maintenance and readability, cost 20% in runtime. Going the other way, having i64() call another function i64a() which does the work, common when using wrapper functions, it took 60 seconds. A 30% slowdown, even though most of the work is doing numeric shifts and adds. So function calls do seem to be expensive operations in CPython, and any benchmarks which highlight that fact should be welcomed. (Note than none of this seems to affect PyPy, which ran at the same speed with i64(), without it, or with both i64() and i64a().) -- bartc From bc at freeuk.com Mon Feb 26 15:32:29 2018 From: bc at freeuk.com (bartc) Date: Mon, 26 Feb 2018 20:32:29 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <20180223033128.GB10273@bladeshadow.org> <20180223163241.GC10273@bladeshadow.org> <20180223184306.GE10273@bladeshadow.org> <700aebe6-0d27-43c0-a05c-f64ad75e50f3@googlegroups.com> Message-ID: <89_kC.293064$Tz3.124739@fx25.am4> On 26/02/2018 20:27, bartc wrote: > On 26/02/2018 19:50, Chris Angelico wrote: >> On Tue, Feb 27, 2018 at 6:37 AM, Rick Johnson > >>> So what? Latency is latency. And whether it occurs over the >>> course of one heavily recursive algorithm that constitutes >>> the depth and breadth of an entire program (a la fib()), or >>> it is the incremental cumulative consequence of the entire >>> program execution, the fact remains that function call >>> overhead contributes to a significant portion of the latency >>> inherent in some trivial, and *ALL* non-trivial, modern >>> software. [Sorry, the bit of your (Chris') post I replied to got chopped by mistake. Here is my post again with the right context:] CA: > By saying "the fact remains", you're handwaving away the work of > actually measuring that function call overhead is "significant". Can > you show me those numbers? Steve's point is that it is NOT > significant, because non-toy functions have non-trivial bodies. If you > wish to disagree, you have to demonstrate that the function call is > *itself* costly, even when there's a real body to it. > Take the last bit of Python I posted, which was that RNG test. It uses this function: def i64(x): return x & 0xFFFFFFFFFFFFFFFF This is a small function, but it can't be a toy one as it was suggested by Ned Batchelder. And the test program wasn't at all recursive. Running the program with N=10_000_000 took 46 seconds. Replacing the i64() call with '&m' (where m is a global set to 0xFFFFFFFFFFFFFFFF), it took 38 seconds. So putting that fix into a function, which is convenient for coding, maintenance and readability, cost 20% in runtime. Going the other way, having i64() call another function i64a() which does the work, common when using wrapper functions, it took 60 seconds. A 30% slowdown, even though most of the work is doing numeric shifts and adds. So function calls do seem to be expensive operations in CPython, and any benchmarks which highlight that fact should be welcomed. (Note than none of this seems to affect PyPy, which ran at the same speed with i64(), without it, or with both i64() and i64a().) -- bartc From rantingrickjohnson at gmail.com Mon Feb 26 16:12:20 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 26 Feb 2018 13:12:20 -0800 (PST) Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: <5ca501bb-189f-41f0-b0f5-8abaabce13e6@googlegroups.com> On Monday, February 26, 2018 at 8:44:14 AM UTC-6, Steven D'Aprano wrote: > I have a class with a large number of parameters (about > ten) assigned in `__init__`. The class then has a number of > methods which accept *optional* arguments with the same > names as the constructor/initialiser parameters. If those > arguments are None, the defaults are taken from the > instance attributes. > > An example might be something like this: > > class Foo: > def __init__(self, bashful, doc, dopey, grumpy, > happy, sleepy, sneezy): > self.bashful = bashful # etc > > def spam(self, bashful=None, doc=None, dopey=None, > grumpy=None, happy=None, sleepy=None, > sneezy=None): > if bashful is None: > bashful = self.bashful > if doc is None: > doc = self.doc > if dopey is None: > dopey = self.dopey > if grumpy is None: > grumpy = self.grumpy > if happy is None: > happy = self.happy > if sleepy is None: > sleepy = self.sleepy > if sneezy is None: > sneezy = self.sneezy > # now do the real work... > > def eggs(self, bashful=None, # etc... > ): > if bashful is None: > bashful = self.bashful > # and so on Steven... even if this example code is absolutely atrocious (because it is!), and even *IF* i have a uneasy suspicion that this "inquiry" masks some dastardly malicious plan (because i do!), I must admit, I am happy to see that you are _finally_ embracing the OOP paradigm. For starters, I would suggest replacing those ugly if- clauses with some short-circuit or'd-logic. But obviously you need to run that "paragraph of arguments" through a single helper function, as the point of "DRY" is "Don't Repeat Yourself". From ian.g.kelly at gmail.com Mon Feb 26 16:37:11 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 14:37:11 -0700 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: On Mon, Feb 26, 2018 at 1:09 PM, wrote: > def foo(self, *args, **kwargs): > assert len(args) == 0 Better: def foo(self, **kwargs): > So, use the inspect module to detect the valid arguments > from the class initializer. Then use **kwargs in every > class method. It would be nice if the full method signature > can be kept, but I have not yet figured out if this is > possible. https://pypi.python.org/pypi/decorator/4.2.1 From drsalists at gmail.com Mon Feb 26 16:40:14 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Mon, 26 Feb 2018 13:40:14 -0800 Subject: matrix multiplication In-Reply-To: <87efl7pu8t.fsf@gmail.com> References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> Message-ID: On Mon, Feb 26, 2018 at 8:53 AM, Seb wrote: > On Sun, 25 Feb 2018 18:52:14 -0500, > Terry Reedy wrote: > > [...] > >> numpy has a matrix multiply function and now the '@' matrix multiply >> operator. > > Yes, but what I was wondering is whether there's a faster way of > multiplying each row (1x3) of a matrix by another matrix (3x3), compared > to looping through the matrix row by row as shown in the code. Perhaps > I'm not understanding how to use the broadcasting features of `matmul`. I'm a bit of a numpy newb, but this sounds like numpy's "broadcasting" feature: http://lmgtfy.com/?q=numpy+broadcasting From ian.g.kelly at gmail.com Mon Feb 26 16:43:22 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 14:43:22 -0700 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: On Mon, Feb 26, 2018 at 2:37 PM, Ian Kelly wrote: > On Mon, Feb 26, 2018 at 1:09 PM, wrote: >> def foo(self, *args, **kwargs): >> assert len(args) == 0 > > Better: > > def foo(self, **kwargs): > >> So, use the inspect module to detect the valid arguments >> from the class initializer. Then use **kwargs in every >> class method. It would be nice if the full method signature >> can be kept, but I have not yet figured out if this is >> possible. > > https://pypi.python.org/pypi/decorator/4.2.1 Also, in Python 3.4+ you can just create a Signature object (either from scratch or by calling inspect.signature()) and then assign it to the function's __signature__ attribute. From ian.g.kelly at gmail.com Mon Feb 26 17:07:44 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 15:07:44 -0700 Subject: matrix multiplication In-Reply-To: References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> Message-ID: On Mon, Feb 26, 2018 at 2:40 PM, Dan Stromberg wrote: > On Mon, Feb 26, 2018 at 8:53 AM, Seb wrote: >> On Sun, 25 Feb 2018 18:52:14 -0500, >> Terry Reedy wrote: >> >> [...] >> >>> numpy has a matrix multiply function and now the '@' matrix multiply >>> operator. >> >> Yes, but what I was wondering is whether there's a faster way of >> multiplying each row (1x3) of a matrix by another matrix (3x3), compared >> to looping through the matrix row by row as shown in the code. Perhaps >> I'm not understanding how to use the broadcasting features of `matmul`. > > I'm a bit of a numpy newb, but this sounds like numpy's "broadcasting" feature: > http://lmgtfy.com/?q=numpy+broadcasting Taking LMGTFY to a whole new level of rudeness by obviously not even bothering to read the entire paragraph before responding. From drsalists at gmail.com Mon Feb 26 17:12:26 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Mon, 26 Feb 2018 14:12:26 -0800 Subject: matrix multiplication In-Reply-To: References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> Message-ID: On Mon, Feb 26, 2018 at 2:07 PM, Ian Kelly wrote: > On Mon, Feb 26, 2018 at 2:40 PM, Dan Stromberg wrote: >> On Mon, Feb 26, 2018 at 8:53 AM, Seb wrote: >>> On Sun, 25 Feb 2018 18:52:14 -0500, >>> Terry Reedy wrote: >>> >>> [...] >>> >>>> numpy has a matrix multiply function and now the '@' matrix multiply >>>> operator. >>> >>> Yes, but what I was wondering is whether there's a faster way of >>> multiplying each row (1x3) of a matrix by another matrix (3x3), compared >>> to looping through the matrix row by row as shown in the code. Perhaps >>> I'm not understanding how to use the broadcasting features of `matmul`. >> >> I'm a bit of a numpy newb, but this sounds like numpy's "broadcasting" feature: >> http://lmgtfy.com/?q=numpy+broadcasting > > Taking LMGTFY to a whole new level of rudeness by obviously not even > bothering to read the entire paragraph before responding. Is LMGTFY rude? I think maybe it was back when it said "Now, was that so hard?", but today there's no snarkiness to be found on the site, AFAIK. Also, like I said, I'm a numpy newbie. If I misunderstood the question, that's probably why. From ben+python at benfinney.id.au Mon Feb 26 17:15:12 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 27 Feb 2018 09:15:12 +1100 Subject: Is there are good DRY fix for this painful design pattern? References: Message-ID: <85tvu39z4f.fsf@benfinney.id.au> Steven D'Aprano writes: > I have a class with a large number of parameters (about ten) assigned > in `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the > constructor/initialiser parameters. If those arguments are None, the > defaults are taken from the instance attributes. The catalogue in Refactoring identifies what you need: Introduce Parameter Object You have a group of parameters that naturally go together. Replace them with an object. Applying that to your example:: class Foo: def __init__(self, wibble_param, dwarves, wobble_param): do_stuff_with(wibble_param) self.dwarves = dwarves do_other_stuff_with(wobble_param) def spam(self, dwarves=None): if dwarves is None: dwarves = self.dwarves # ? now do the real spam work ? def eggs(self, dwarves=None): if dwarves is None: dwarves = self.dwarves # ? now do the real eggs work ? > [?] to add insult to injury the class is still under active > development with an unstable API, so every time I change one of the > parameters, or add a new one, I have to change it in over a dozen > places. By encapsulating those parameters that all belong together, you can change that set of parameters (by changing what ?dwarves? gets passed around) and the function signatures don't change. -- \ ?All opinions are not equal. Some are a very great deal more | `\ robust, sophisticated, and well supported in logic and argument | _o__) than others.? ?Douglas Adams | Ben Finney From ian.g.kelly at gmail.com Mon Feb 26 17:17:53 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 15:17:53 -0700 Subject: matrix multiplication In-Reply-To: References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> Message-ID: On Mon, Feb 26, 2018 at 3:12 PM, Dan Stromberg wrote: > On Mon, Feb 26, 2018 at 2:07 PM, Ian Kelly wrote: >> Taking LMGTFY to a whole new level of rudeness by obviously not even >> bothering to read the entire paragraph before responding. > > Is LMGTFY rude? I think maybe it was back when it said "Now, was that > so hard?", but today there's no snarkiness to be found on the site, > AFAIK. Maybe it's gotten less condescending since the last time I followed such a link. It is notably disallowed on Stack Overflow for the reason of being considered impolite. From ziggyvimar at gmail.com Mon Feb 26 18:23:55 2018 From: ziggyvimar at gmail.com (ziggyvimar at gmail.com) Date: Mon, 26 Feb 2018 15:23:55 -0800 (PST) Subject: help me ? In-Reply-To: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On Monday, February 26, 2018 at 9:40:54 AM UTC, Negru Popi wrote: > Define 2 lists. The first one must contain the integer values 1, 2 and 3 and the second one the string values a, b and c. Iterate through both lists to create another list that contains all the combinations of the A and B elements. The final list should look like one of the 2 lists: > 1. [1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c] > 2. [a1, a2. a3, b1, b2, b3, c1, c2, c3] > BONUS: Make the final list contain all possible combinations : [1a, a1, 1b, b1, 1c, c1, 2a, a2, 2b, b2, 2c, c2, 3a, a3, 3b, b3, 3c, c3] > > Help me ! Homework? From greg.ewing at canterbury.ac.nz Mon Feb 26 18:25:30 2018 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 27 Feb 2018 12:25:30 +1300 Subject: matrix multiplication In-Reply-To: References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> Message-ID: Seb wrote: > I was wondering is whether there's a faster way of > multiplying each row (1x3) of a matrix by another matrix (3x3), compared > to looping through the matrix row by row as shown in the code. Just multiply the two matrices together. If A is an nx3 matrix and B is a 3x3 matrix, then C = A @ B is an nx3 matrix where C[i] = A[i] @ B. (This is a property of matrix multiplication in general, nothing special about numpy.) -- Greg From python at mrabarnett.plus.com Mon Feb 26 18:43:53 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 26 Feb 2018 23:43:53 +0000 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: <5ca501bb-189f-41f0-b0f5-8abaabce13e6@googlegroups.com> References: <5ca501bb-189f-41f0-b0f5-8abaabce13e6@googlegroups.com> Message-ID: <6bd7aeb9-cbf5-827b-5449-852c3ae14516@mrabarnett.plus.com> On 2018-02-26 21:12, Rick Johnson wrote: > On Monday, February 26, 2018 at 8:44:14 AM UTC-6, Steven D'Aprano wrote: >> I have a class with a large number of parameters (about >> ten) assigned in `__init__`. The class then has a number of >> methods which accept *optional* arguments with the same >> names as the constructor/initialiser parameters. If those >> arguments are None, the defaults are taken from the >> instance attributes. >> >> An example might be something like this: >> >> class Foo: >> def __init__(self, bashful, doc, dopey, grumpy, >> happy, sleepy, sneezy): >> self.bashful = bashful # etc >> >> def spam(self, bashful=None, doc=None, dopey=None, >> grumpy=None, happy=None, sleepy=None, >> sneezy=None): >> if bashful is None: >> bashful = self.bashful >> if doc is None: >> doc = self.doc >> if dopey is None: >> dopey = self.dopey >> if grumpy is None: >> grumpy = self.grumpy >> if happy is None: >> happy = self.happy >> if sleepy is None: >> sleepy = self.sleepy >> if sneezy is None: >> sneezy = self.sneezy >> # now do the real work... >> >> def eggs(self, bashful=None, # etc... >> ): >> if bashful is None: >> bashful = self.bashful >> # and so on > > > Steven... even if this example code is absolutely atrocious > (because it is!), and even *IF* i have a uneasy suspicion > that this "inquiry" masks some dastardly malicious plan > (because i do!), I must admit, I am happy to see that you > are _finally_ embracing the OOP paradigm. > > For starters, I would suggest replacing those ugly if- > clauses with some short-circuit or'd-logic. But obviously > you need to run that "paragraph of arguments" through a > single helper function, as the point of "DRY" is "Don't > Repeat Yourself". > Before using or'd-logic, you need to know whether the value could be falsey, e.g. 0. > > From tjreedy at udel.edu Mon Feb 26 18:57:21 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 26 Feb 2018 18:57:21 -0500 Subject: Questions about `locals` builtin In-Reply-To: References: Message-ID: On 2/26/2018 1:55 PM, Kirill Balunov wrote: > Hi, > > I am a little bit confused with `locals` builtin in these moments: > > 1. The documentation says that _free varaibles_ are returned, which seems > incorrect description. In my mind the term free variable refers to > variables used in a function that are not local variables nor parameters of > that function. I disagree with the terminology used in this section of the docs, so I won't defend or explain. The important points are that 1. *Python* code is that executed in the context of multiple directly accessible namespaces; 2. there is more than one way to implement a namespace; 3. the Python namespace system has grown in complexity since the beginning; and 4. some details depend on the implementation (CPython versus others) and Python and therefore implementation version. The point of point 3 is that terminology and details would likely be different if Python were freshly designed more or less as it is today, and some things only make more or less sense in historical context. Learn what you need to know to write code that works. The program namespace is called 'builtins'. Treat it as readonly. The namespace, as opposed to its contents, cannot be accessed unless imported. Each module namespace is accessed as globals(). It must be a dict. The names of builtins are accessed as it they were in globals, even though, in CPython, they are not. (I believe 'globals' predates multi-module programs.) The local namespace is the default recipient of name bindings from assignment, class, and def statements. At module scope, the local namespace is the module or global namespace. A binding in module locals == globals can mask a builtin. 'def print(): pass'. Deletion of that binding unmasks the builtin. 'del print'. (If an implementation actually initialized each globals() by copying builtins, then each deletion would have to check for possible unmasking.) Class statements get a new local namespace, which defaults to some sort of mapping. When the class statement finishes, the local names become class attributes. Be default, function local namespaces in CPython are implemented as arrays, with local names compiles as indexes in the array. Calling surrounding function local names collectively 'nonlocals' is the result of months of bikeshedding. -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Mon Feb 26 19:17:10 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 00:17:10 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Tue, 27 Feb 2018 02:09:53 +1100, Chris Angelico wrote: > You're still reimplementing the C code in Python, which is inefficient. > Have you considered going back to the *actual algorithm* and > implementing that idiomatically in Python? I think you'll find that (a) > the code is more readable, and (b) the run time is much lower. Chris, I think this is incredibly optimistic, if not naive. We're talking about a PRNG by Marsaglia, so my guess is that the "original algorithm" *is* the C code. Or possibly Fortran. Even if not, even if there is actually a language-neutral algorithm, its a PRNG which means its going to be filled with bit-twiddling and number- crunching operations. Pure Python without a JIT is never going to be competitive with C or Fortran for code like that. -- Steve From rantingrickjohnson at gmail.com Mon Feb 26 19:23:39 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 26 Feb 2018 16:23:39 -0800 (PST) Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: <5ca501bb-189f-41f0-b0f5-8abaabce13e6@googlegroups.com> <6bd7aeb9-cbf5-827b-5449-852c3ae14516@mrabarnett.plus.com> Message-ID: <1383fddf-4faa-4a44-b08e-a5283a738072@googlegroups.com> On Monday, February 26, 2018 at 5:44:18 PM UTC-6, MRAB wrote: [...] > Before using or'd-logic, you need to know whether the value > could be falsey, e.g. 0. True. However. Steven failed to provide any info that might help us determine the types of these parameters, and as such, i was forced to take the liberty of assumption. Now, while it's quite possible someone could assign dwarfy-named- symbols to implicit or explicit boolean values, i find it highly unlikely, and totally unintuitive. But then again, i have my suspicions about ths thread. ;-) From steve+comp.lang.python at pearwood.info Mon Feb 26 19:25:53 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 00:25:53 +0000 (UTC) Subject: Questions about `locals` builtin References: Message-ID: On Mon, 26 Feb 2018 21:55:35 +0300, Kirill Balunov wrote: > Hi, > > I am a little bit confused with `locals` builtin in these moments: > > 1. The documentation says that _free varaibles_ are returned, which > seems incorrect description. I can't answer that, sorry. > 2. The documentation has a note that "The contents of this dictionary > should not be modified". Which implies that it is a read only mapping. > So the question why it is `dict` instead of `types.MappingProxyType`? Mostly because locals() predates MappingProxyType by many years, and also because that restriction doesn't apply to other implementations of Python such as Jython and IronPython. In CPython, the dict returned by locals() is a copy of the local namespace, so modifying the dict doesn't modify the real local variables. (Well, sometimes it does, but not always. The story in Python 2 is really complex.) But in Jython and IronPython, it actually is the namespace, so writing to it works like writing to globals. So writing to locals *sometimes* works, and cannot be prohibited outright, but if you want portable code, you have to avoid it. > 3. There is one more moment: local variables had been determined when > function was compiled. But `locals` returns _some_ current runtime copy. Have you tried it in Python 2 or 3? The behaviour may be less confusing in 3. -- Steve From rosuav at gmail.com Mon Feb 26 19:35:52 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 11:35:52 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: On Tue, Feb 27, 2018 at 11:17 AM, Steven D'Aprano wrote: > On Tue, 27 Feb 2018 02:09:53 +1100, Chris Angelico wrote: > >> You're still reimplementing the C code in Python, which is inefficient. >> Have you considered going back to the *actual algorithm* and >> implementing that idiomatically in Python? I think you'll find that (a) >> the code is more readable, and (b) the run time is much lower. > > Chris, I think this is incredibly optimistic, if not naive. We're talking > about a PRNG by Marsaglia, so my guess is that the "original algorithm" > *is* the C code. Or possibly Fortran. > > Even if not, even if there is actually a language-neutral algorithm, its > a PRNG which means its going to be filled with bit-twiddling and number- > crunching operations. Pure Python without a JIT is never going to be > competitive with C or Fortran for code like that. > I may have been a little unclear. It's highly unlikely that the run time of the properly-implemented Python code will be lower than the original C or Fortran. But it most certainly CAN be more efficient than the Python reimplementation of the C implementation, which would narrow the gap considerably. Implementing Python idiotically instead of idiomatically gives you suboptimal performance. ChrisA From drsalists at gmail.com Mon Feb 26 20:05:46 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Mon, 26 Feb 2018 17:05:46 -0800 Subject: Questions about `locals` builtin In-Reply-To: References: Message-ID: On Mon, Feb 26, 2018 at 4:25 PM, Steven D'Aprano wrote: > On Mon, 26 Feb 2018 21:55:35 +0300, Kirill Balunov wrote: >> 2. The documentation has a note that "The contents of this dictionary >> should not be modified". Which implies that it is a read only mapping. >> So the question why it is `dict` instead of `types.MappingProxyType`? > > Mostly because locals() predates MappingProxyType by many years, and also > because that restriction doesn't apply to other implementations of Python > such as Jython and IronPython. > > In CPython, the dict returned by locals() is a copy of the local > namespace, so modifying the dict doesn't modify the real local variables. > > (Well, sometimes it does, but not always. The story in Python 2 is really > complex.) > > But in Jython and IronPython, it actually is the namespace, so writing to > it works like writing to globals. > > So writing to locals *sometimes* works, and cannot be prohibited > outright, but if you want portable code, you have to avoid it. I find this interesting. So I decided to write a little test program, to see it first hand. I don't have IronPython handy, but according my (quite possibly flawed) test program, locals() is a copy on CPython 3, CPython 2, Pypy3, Pypy, Jython and MicroPython. I didn't see any interpreters that returned the namespace itself. What am I missing? Here's the test program. Perhaps there's something wrong in it? #!/usr/local/cpython-3.6/bin/python3 """Test whether locals() is a copy of the local namespace, or a reference.""" import sys def main(): """Run the test.""" var = 1 locs = locals() locs['var'] = 2 if var == 2: sys.stdout.write('locals not just a copy\n') else: sys.stdout.write('locals is just a copy\n') main() From rantingrickjohnson at gmail.com Mon Feb 26 20:18:38 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 26 Feb 2018 17:18:38 -0800 (PST) Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Tuesday, February 20, 2018 at 5:45:36 PM UTC-6, Steven D'Aprano wrote: > On Tue, 20 Feb 2018 12:42:23 -0800, Rick Johnson wrote: > > > For instance, if the age is queried many times a second, > > it would be a much wiser design to set-up an event that > > will advance the age at the end of the last second of > > every year > > Do you really mean to say that everybody in the world has > their birthday on January 1st? We're not racehorses you > know. No, silly rabbit. I was thinking about the problem from a _relative_ perspective, whereas you were thinking about the problem from a _global_ perspective. Neither perspective is wrong. If you read my exact words again: "a much wiser design to set-up an event that will advance the age at the end of the last second of every year" ...you'll notice that i mentioned no specific date. Therefore, "the last day of the year" (in relativistic terms) is 11:59:59PM on the calendar day which _precedes_ the day of the month for which you were born. So, for instance: if your birthday is January 25th 1969, the last second of the last day of your _first_ year is January 24th 1970 @ 11:59:59PM. And the last second of the last day of your _second_ year is January 24th 1971 @ 11:59:59PM. And so forth... Does this make sense? > > Under your scheme, 99.7% of records will return the wrong > age (off by one) at least once per year. Statistically, 50% > of queries will be wrong. Can you whip-up a small code example which proves your assertion? Hey, and feel free to include the seven dwarfs if you like. ;-) From spluque at gmail.com Mon Feb 26 20:51:59 2018 From: spluque at gmail.com (Seb) Date: Mon, 26 Feb 2018 19:51:59 -0600 Subject: matrix multiplication References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> Message-ID: <87muzvi4hs.fsf@otaria.sebmel.org> On Tue, 27 Feb 2018 12:25:30 +1300, Gregory Ewing wrote: > Seb wrote: >> I was wondering is whether there's a faster way of multiplying each >> row (1x3) of a matrix by another matrix (3x3), compared to looping >> through the matrix row by row as shown in the code. > Just multiply the two matrices together. > If A is an nx3 matrix and B is a 3x3 matrix, then C = A @ B is an nx3 > matrix where C[i] = A[i] @ B. > (This is a property of matrix multiplication in general, nothing > special about numpy.) I think that's only true if B is the same for every row in A. In the code I posted, B varies by row of A. -- Seb From bc at freeuk.com Mon Feb 26 20:57:49 2018 From: bc at freeuk.com (bartc) Date: Tue, 27 Feb 2018 01:57:49 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> Message-ID: <8W2lC.362298$jQ2.204214@fx46.am4> On 27/02/2018 00:35, Chris Angelico wrote: > On Tue, Feb 27, 2018 at 11:17 AM, Steven D'Aprano > wrote: >> On Tue, 27 Feb 2018 02:09:53 +1100, Chris Angelico wrote: >> >>> You're still reimplementing the C code in Python, which is inefficient. >>> Have you considered going back to the *actual algorithm* and >>> implementing that idiomatically in Python? I think you'll find that (a) >>> the code is more readable, and (b) the run time is much lower. >> >> Chris, I think this is incredibly optimistic, if not naive. We're talking >> about a PRNG by Marsaglia, so my guess is that the "original algorithm" >> *is* the C code. Or possibly Fortran. >> >> Even if not, even if there is actually a language-neutral algorithm, its >> a PRNG which means its going to be filled with bit-twiddling and number- >> crunching operations. Pure Python without a JIT is never going to be >> competitive with C or Fortran for code like that. >> > > I may have been a little unclear. It's highly unlikely that the run > time of the properly-implemented Python code will be lower than the > original C or Fortran. But it most certainly CAN be more efficient > than the Python reimplementation of the C implementation, which would > narrow the gap considerably. Implementing Python idiotically instead > of idiomatically gives you suboptimal performance. Nonsense. You shouldn't need to care about such things. An algorithm is an algorithm. And the better ones should be easily written in any language. This particular one, which was of interest because the calculations tended to overflow into undesirable bignums, is just a bunch of calculations. 'Bit-twiddling'. Anyway, even this pure Python version can deliver pseudo random numbers at some 200,000 per second, while the built-in generator does 450,000 per second, so it's not bad going. Of course, the C version will generate them at over 100 million per second. -- bartc From rosuav at gmail.com Mon Feb 26 21:22:13 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 13:22:13 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Tue, Feb 27, 2018 at 12:18 PM, Rick Johnson wrote: > On Tuesday, February 20, 2018 at 5:45:36 PM UTC-6, Steven D'Aprano wrote: >> On Tue, 20 Feb 2018 12:42:23 -0800, Rick Johnson wrote: >> >> > For instance, if the age is queried many times a second, >> > it would be a much wiser design to set-up an event that >> > will advance the age at the end of the last second of >> > every year >> >> Do you really mean to say that everybody in the world has >> their birthday on January 1st? We're not racehorses you >> know. > > > No, silly rabbit. I was thinking about the problem from a > _relative_ perspective, whereas you were thinking about the > problem from a _global_ perspective. Neither perspective is > wrong. > > If you read my exact words again: > > "a much wiser design to set-up an event that will advance > the age at the end of the last second of every year" > > ...you'll notice that i mentioned no specific date. > > Therefore, "the last day of the year" (in relativistic > terms) is 11:59:59PM on the calendar day which _precedes_ > the day of the month for which you were born. > > So, for instance: if your birthday is January 25th 1969, the > last second of the last day of your _first_ year is January > 24th 1970 @ 11:59:59PM. And the last second of the last day > of your _second_ year is January 24th 1971 @ 11:59:59PM. And > so forth... > > Does this make sense? Yes! It does. All you have to do is run a batch job in exactly the very last second of *every single day*. Totally not a problem! Have you ever run a database in your life? Tip: When you're in a hole, stop digging. ChrisA From rosuav at gmail.com Mon Feb 26 21:27:10 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 13:27:10 +1100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: <8W2lC.362298$jQ2.204214@fx46.am4> References: <65185193-a1e5-437d-8679-fd9fb0fca0ba@googlegroups.com> <8W2lC.362298$jQ2.204214@fx46.am4> Message-ID: On Tue, Feb 27, 2018 at 12:57 PM, bartc wrote: > On 27/02/2018 00:35, Chris Angelico wrote: >> >> On Tue, Feb 27, 2018 at 11:17 AM, Steven D'Aprano >> wrote: >>> >>> On Tue, 27 Feb 2018 02:09:53 +1100, Chris Angelico wrote: >>> >>>> You're still reimplementing the C code in Python, which is inefficient. >>>> Have you considered going back to the *actual algorithm* and >>>> implementing that idiomatically in Python? I think you'll find that (a) >>>> the code is more readable, and (b) the run time is much lower. >>> >>> >>> Chris, I think this is incredibly optimistic, if not naive. We're talking >>> about a PRNG by Marsaglia, so my guess is that the "original algorithm" >>> *is* the C code. Or possibly Fortran. >>> >>> Even if not, even if there is actually a language-neutral algorithm, its >>> a PRNG which means its going to be filled with bit-twiddling and number- >>> crunching operations. Pure Python without a JIT is never going to be >>> competitive with C or Fortran for code like that. >>> >> >> I may have been a little unclear. It's highly unlikely that the run >> time of the properly-implemented Python code will be lower than the >> original C or Fortran. But it most certainly CAN be more efficient >> than the Python reimplementation of the C implementation, which would >> narrow the gap considerably. Implementing Python idiotically instead >> of idiomatically gives you suboptimal performance. > > > Nonsense. You shouldn't need to care about such things. An algorithm is an > algorithm. And the better ones should be easily written in any language. > > This particular one, which was of interest because the calculations tended > to overflow into undesirable bignums, is just a bunch of calculations. > 'Bit-twiddling'. > > Anyway, even this pure Python version can deliver pseudo random numbers at > some 200,000 per second, while the built-in generator does 450,000 per > second, so it's not bad going. The built-in generator is using a completely different algorithm though, so rate of generation isn't the entire story. How long is the period of the one you're using? (How long before it loops?) If you churn the generator to an arbitrary number and then take the next 100 values it generates, are they randomly distributed? Can you reconstruct the RNG's internal state by watching it generate numbers for a short while? Obviously no PRNG is going to be perfect at this, but there are certainly degrees of quality to be compared. ChrisA From steve+comp.lang.python at pearwood.info Mon Feb 26 22:09:56 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 03:09:56 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Mon, 26 Feb 2018 17:18:38 -0800, Rick Johnson wrote: > On Tuesday, February 20, 2018 at 5:45:36 PM UTC-6, Steven D'Aprano > wrote: >> On Tue, 20 Feb 2018 12:42:23 -0800, Rick Johnson wrote: >> >> > For instance, if the age is queried many times a second, it would be >> > a much wiser design to set-up an event that will advance the age at >> > the end of the last second of every year >> >> Do you really mean to say that everybody in the world has their >> birthday on January 1st? We're not racehorses you know. [...] > If you read my exact words again: > > "a much wiser design to set-up an event that will advance the age at > the end of the last second of every year" Oh, you mean the exact same words already quoted above? > ...you'll notice that i mentioned no specific date. Yes you did: "the last second of every year" is always 23:59:59 of 31st December, and it is always the same time and date "every year". Just because you didn't explicitly state the name of a month doesn't mean you didn't implicitly specify one. Perhaps you didn't *intend* to specify the last day of the year, in which case you ought to just accept that your writing wasn't as clear as you intended, instead of blaming others for failing to read your mind. > Therefore, "the last day of the year" (in relativistic terms) is > 11:59:59PM on the calendar day which _precedes_ the day of the month for > which you were born. Relativistic terms? You mean when travelling at 90% of the speed of light or in close orbit around a neutron star? There are those language skills letting you down again :-P I think you mean *relative* terms. [...] >> Under your scheme, 99.7% of records will return the wrong age (off by >> one) at least once per year. Statistically, 50% of queries will be >> wrong. > > > Can you whip-up a small code example which proves your assertion? Hey, > and feel free to include the seven dwarfs if you like. ;-) Under the racehorse scheme, 1/365 of your data will be born on the 1st of Jan, 1/365 on the 2nd Jan, and so on through 31st Dec. (Ignoring leap years.) Half of the data records will be updated early, and half will be updated late. Only those born on New Years Day (about 0.3% of the records) will be updated on the correct day. Hence 99.7% will be wrong for at least one day per year. If queries are made randomly, they'll be off by one 50% of the time. Why 50%? 1/365 of your records will never be wrong, 1/365 will be wrong one day of the year, 1/365 will be wrong two days of the year, 1/365 will be wrong three days of the year, ... 1/365 will be wrong 364 days of the year. So on average, records are wrong 182 days of the year, which is close enough to half of 365. So if you query a random record on a random day, there's close enough to a fifty percent chance it will be off by one. Now that I understand that you didn't intend this racehorse scheme, I'm not going to bother writing a simulation. But it is just simple statistics, assuming that the actual birthday is distributed randomly across the year, and assuming that queries for records come in at random times. (And to be pedantic, I mean a uniform random distribution, and that queries and birthdays are independent.) -- Steve From steve+comp.lang.python at pearwood.info Mon Feb 26 22:25:35 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 03:25:35 +0000 (UTC) Subject: Are the critiques in "All the things I hate about Python" valid? References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Mon, 26 Feb 2018 17:18:38 -0800, Rick Johnson wrote: [...] > So, for instance: if your birthday is January 25th 1969, the last second > of the last day of your _first_ year is January 24th 1970 @ 11:59:59PM. > And the last second of the last day of your _second_ year is January > 24th 1971 @ 11:59:59PM. And so forth... > > Does this make sense? Indeed it does, and frankly, the Racehorse scheme is better. At least with the Racehorse scheme, you only need to update the database once a year, at midnight on the new year, which hopefully is the quietest time of the year for you. You can lock access to the database, run the update, and hopefully be up and running again before anyone notices anything other than a minor outage. With your scheme, well, I can think of a few ways to do it, none of which are good. A database expert might be able to think of some better ideas, but you might: 1. Run a separate scheduled job for each record, which does nothing but advance the age by one at a certain time, then sleep for a year. If you have ten million records, you need ten million scheduled jobs; I doubt many scheduling systems can cope with that many jobs. (But I welcome correction.) Also, few scheduling systems guarantee that jobs will execute at *precisely* the time you expect. If the system is down at the time the job was scheduled to run, they may never run at all. So there is likely to be a lag between when you want the records updated, and when they actually are updated. No, using scheduled jobs is fragile, and expensive. Plan 2: have a single job that does nothing but scan the database, continuously in a loop, and if a record's birthdate is more than a year in the past, and hasn't been updated in the last year, update the age by one. Actually, I think this sucks worse than the ten-million-scheduled-jobs idea. Hopefully it will be obvious why this idea is so awful. Plan 3: have a trigger that runs whenever a record is queried or accessed. If the birthdate is more than a year in the past, and it has been more than a year since the last access, then update the age. This at least doesn't *entirely* suck. But if you're going to go to the trouble of doing this on *every* access to the record, isn't it simpler to just make the age a computed field that calculates the age when needed? The cost of computing the age is not that expensive, especially if you store the birthdate in seconds. It's just a subtraction, maybe followed by a division if you want the age in years. It hardly seems worthwhile storing the age as a pre-computed integer if you then need a cunning scheme to possibly update that integer on every access to the record. I think that Rick's "optimization" here is a perfect example of pessimisation (making a program slower in the mistaken belief that you're making it faster). To quote W.A. Wulf: "More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason ? including blind stupidity." -- Steve From ian.g.kelly at gmail.com Mon Feb 26 22:55:36 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 20:55:36 -0700 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Mon, Feb 26, 2018 at 8:09 PM, Steven D'Aprano wrote: > Yes you did: "the last second of every year" is always 23:59:59 of 31st > December, and it is always the same time and date "every year". Except when it's 23:59:60 or 23:59:61 (which hasn't yet happened but could). From rosuav at gmail.com Mon Feb 26 23:22:58 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 27 Feb 2018 15:22:58 +1100 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Tue, Feb 27, 2018 at 2:55 PM, Ian Kelly wrote: > On Mon, Feb 26, 2018 at 8:09 PM, Steven D'Aprano > wrote: >> Yes you did: "the last second of every year" is always 23:59:59 of 31st >> December, and it is always the same time and date "every year". > > Except when it's 23:59:60 or 23:59:61 (which hasn't yet happened but could). Unless you use a leap smear to avoid it. ChrisA From ian.g.kelly at gmail.com Mon Feb 26 23:23:22 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 26 Feb 2018 21:23:22 -0700 Subject: Are the critiques in "All the things I hate about Python" valid? In-Reply-To: References: <4qqiC.223282$kD1.96803@fx34.am4> <7wziC.125009$BM3.115116@fx14.am4> <30so8dduinfoc0mgbmci8ua85cds2o5s5j@4ax.com> <54a575b7-3472-a5a3-59f6-504e951be9ae@mrabarnett.plus.com> Message-ID: On Mon, Feb 26, 2018 at 8:55 PM, Ian Kelly wrote: > On Mon, Feb 26, 2018 at 8:09 PM, Steven D'Aprano > wrote: >> Yes you did: "the last second of every year" is always 23:59:59 of 31st >> December, and it is always the same time and date "every year". > > Except when it's 23:59:60 or 23:59:61 (which hasn't yet happened but could). Actually, after doing some reading on the subject just now, I learned that double leap seconds don't actually exist! They were accidentally invented by the drafters of the ANSI C standard and later propagated to the POSIX standard and the ANSI SQL standard. https://www.ucolick.org/~sla/leapsecs/timescales.html#UTC So the actual last second of the year must always be one of 23:59:58, 23:59:59 or 23:59:60. (Hint: although we've never had a negative leap second to date, don't schedule your database update for 23:59:59 if you want to ensure that it actually happens.) From steve+comp.lang.python at pearwood.info Mon Feb 26 23:46:57 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 04:46:57 +0000 (UTC) Subject: Is there are good DRY fix for this painful design pattern? References: Message-ID: On Mon, 26 Feb 2018 17:39:43 +0100, Peter Otten wrote: [...] > I have not yet looked into dataclasses. Don't they handle the __init__() > part? Anyway, here's my attempt to make spam() less spammy: I'm not too concerned about __init__, it's only one method :-) > $ cat attrs_to_args_decorator.py > import functools > import inspect > > def add_defaults(f): > argnames = inspect.getfullargspec(f).args[1:] > > @functools.wraps(f) > def wrapper(self, *args, **kw): > args = [ > getattr(self, name) if value is None else value for name, > value in zip(argnames, args) > ] > for name in argnames[len(args):]: > if name not in kw or kw[name] is None: > kw[name] = getattr(self, name) > return f(self, *args, **kw) > return wrapper [...] Interesting, this does look promising, thanks. -- Steve From steve+comp.lang.python at pearwood.info Mon Feb 26 23:51:11 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 04:51:11 +0000 (UTC) Subject: Is there are good DRY fix for this painful design pattern? References: <5ca501bb-189f-41f0-b0f5-8abaabce13e6@googlegroups.com> <6bd7aeb9-cbf5-827b-5449-852c3ae14516@mrabarnett.plus.com> <1383fddf-4faa-4a44-b08e-a5283a738072@googlegroups.com> Message-ID: On Mon, 26 Feb 2018 16:23:39 -0800, Rick Johnson wrote: > On Monday, February 26, 2018 at 5:44:18 PM UTC-6, MRAB wrote: [...] >> Before using or'd-logic, you need to know whether the value could be >> falsey, e.g. 0. > > True. However. Steven failed to provide any info that might help us > determine the types of these parameters, and as such, i was forced to > take the liberty of assumption. Someone held a gun to your head? > Now, while it's quite possible someone > could assign dwarfy-named- symbols to implicit or explicit boolean > values, i find it highly unlikely, and totally unintuitive. But then > again, i have my suspicions about ths thread. ;-) You mean things like passing 0 as an int argument, or "" as a string argument? Yeah, I see what you mean -- that would be such a weird thing to do... /s -- Steve From steve+comp.lang.python at pearwood.info Mon Feb 26 23:58:08 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 04:58:08 +0000 (UTC) Subject: Is there are good DRY fix for this painful design pattern? References: <85tvu39z4f.fsf@benfinney.id.au> Message-ID: On Tue, 27 Feb 2018 09:15:12 +1100, Ben Finney wrote: > Steven D'Aprano writes: > >> I have a class with a large number of parameters (about ten) assigned >> in `__init__`. The class then has a number of methods which accept >> *optional* arguments with the same names as the constructor/initialiser >> parameters. If those arguments are None, the defaults are taken from >> the instance attributes. > > The catalogue in Refactoring identifies what you need: > > Introduce Parameter Object > > You have a group of parameters that naturally go together. > > Replace them with an object. > > Interesting, I'll certainly have a good read of that, thanks. But I fear I mislead you: you seem to have understood that either all the parameters are None, or none of them are, whereas what I meant was that they can vary independently of each other. So it isn't enough to check whether the Parameter object is None, I'd need to check each field within the object. On the other hand, at least I could put the logic of that inside the Parameter object, instead of repeating it everywhere it is needed. On the gripping hand, I *think* that means the caller is responsible for creating and populating the Parameter object, instead of just passing keyword arguments... let me think about that one. Either way, thanks for the suggestion. -- Steve From ben+python at benfinney.id.au Tue Feb 27 00:36:59 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 27 Feb 2018 16:36:59 +1100 Subject: Is there are good DRY fix for this painful design pattern? References: <85tvu39z4f.fsf@benfinney.id.au> Message-ID: <85efl79eo4.fsf@benfinney.id.au> Steven D'Aprano writes: > Interesting, I'll certainly have a good read of that, thanks. But I > fear I mislead you: you seem to have understood that either all the > parameters are None, or none of them are, whereas what I meant was > that they can vary independently of each other. I think that issue ? independent defaults for the values ? is orthogonal. For avoiding the large set of parameters repeated everywhere, you evidently consider that cluster of values a single concept, to be passed around as a unit. That's what the ?Introduce Parameter Object? refactor operation is for: Now that you recognise this cluster is an implicit conceptual unit that different methods share, define an explicit type for it, and pass around an instance of that between the functions. > So it isn't enough to check whether the Parameter object is None, I'd > need to check each field within the object. No problem. You can do that within the methods, as before. Alternatively, you can define it as behaviour of the type. The ?Introduce Parameter Object? refactor leaves that and other options open, because it's an unrelated issue. > On the gripping hand, I *think* that means the caller is responsible > for creating and populating the Parameter object, instead of just > passing keyword arguments... let me think about that one. One good effect of this refactor is you're forced to explicitly *name* this concept of cluster-of-values-with-some-specific-meaning. Once you have a name ? which will confont you with a decision about the meaning of this specific cluster of values ? you then get to thinking about how the values should be initialised, etc. All of that is hidden from the methods that just want to pass around the cluster of values to each other. You then get to freely change it while development continues. > Either way, thanks for the suggestion. Happy hacking. -- \ ?I think it would be a good idea.? ?Mohandas K. Gandhi (when | `\ asked what he thought of Western civilization) | _o__) | Ben Finney From dieter at handshake.de Tue Feb 27 01:52:46 2018 From: dieter at handshake.de (dieter) Date: Tue, 27 Feb 2018 07:52:46 +0100 Subject: Questions about `locals` builtin References: Message-ID: <87inaivs8x.fsf@handshake.de> Kirill Balunov writes: > I am a little bit confused with `locals` builtin in these moments: > > 1. The documentation says that _free varaibles_ are returned, which seems > incorrect description. In my mind the term free variable refers to > variables used in a function that are not local variables nor parameters of > that function. > > In docs: "Update and return a dictionary representing the current local > symbol table. Free variables are returned by `locals()` when it is called > in function blocks, but not in class blocks." I agree with your definition of "free variable" and I think that the documentation partially speaks of it. In a function body, you have two kinds of "free variables": implicitly used free variables and explicitly declared (via "global", "nonlocal") free variables. I think the documentation wants to make clear, that "locals" does not treat "free variables" in a completely consistent way (there is a difference between class and function blocks). I assume that "locals" may return some (but not necessary all) free variables in a function block -- depending on the actual implementation. From auriocus at gmx.de Tue Feb 27 03:08:39 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 27 Feb 2018 09:08:39 +0100 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <8W2lC.362298$jQ2.204214@fx46.am4> Message-ID: Am 27.02.18 um 03:27 schrieb Chris Angelico: > On Tue, Feb 27, 2018 at 12:57 PM, bartc wrote: >> >> Anyway, even this pure Python version can deliver pseudo random numbers at >> some 200,000 per second, while the built-in generator does 450,000 per >> second, so it's not bad going. > > The built-in generator is using a completely different algorithm > though, so rate of generation isn't the entire story. How long is the > period of the one you're using? (How long before it loops?) If you > churn the generator to an arbitrary number and then take the next 100 > values it generates, are they randomly distributed? George Marsaglia is a researcher who invented a couple of PRNGs which are both simple (one of the first was called KISS) yet surprisingly good. See here: https://en.wikipedia.org/wiki/KISS_(algorithm) They possess all these desirable properties you mention above and pass the DIEHARD random number generator tests (also invented by Marsaglia). > Can you > reconstruct the RNG's internal state by watching it generate numbers > for a short while? https://eprint.iacr.org/2011/007.pdf > Obviously no PRNG is going to be perfect at this, but there are > certainly degrees of quality to be compared. It is a field of research and the generator shown by Bart is one of the better generators constructed by an expert in the field. Christian From steve+comp.lang.python at pearwood.info Tue Feb 27 03:13:48 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 08:13:48 +0000 (UTC) Subject: Questions about `locals` builtin References: Message-ID: On Mon, 26 Feb 2018 17:05:46 -0800, Dan Stromberg wrote: [...] > I don't have IronPython handy, but according my (quite possibly flawed) > test program, locals() is a copy on CPython 3, CPython 2, Pypy3, Pypy, > Jython and MicroPython. > > I didn't see any interpreters that returned the namespace itself. > > What am I missing? Interesting... it seems that Jython is less consistent than I remembered. Here is my test function: def test(): y = 2 ns = locals() ns['x'] = 1 ns['y'] = 1 print(x, y) return # Fool the compiler into treating x as a local. if False: x = 999 In Jython 2.5, it prints (1, 2). So assigning to locals creates a new local variable (x) but doesn't update an existing one (y). In CPython, I get: UnboundLocalError: local variable 'x' referenced before assignment in versions 3.5, 2.7, 2,5, 2,4 and even 1.5 (it's just a regular NameError in 1.5). I broke my IronPython installation, but my recollection is that it would print (1, 1). (That's what I thought Jython did too, but I was mistaken.) So correction is noted: in Jython, at least, locals() is not the namespace itself, but some sort of weird proxy to it. -- Steve From zhangyangyu0614 at gmail.com Tue Feb 27 03:34:28 2018 From: zhangyangyu0614 at gmail.com (Xiang Zhang) Date: Tue, 27 Feb 2018 00:34:28 -0800 (PST) Subject: In Python2, does it need to wrap imp.find/load_module with imp_acquire/release_lock? Message-ID: <71381ce7-213b-460e-b5d2-07337158d1ff@googlegroups.com> Just like the title. It seems to me it is needed from the source code but codes in stdlib all doesn't do that. From wolfgang.maier at biologie.uni-freiburg.de Tue Feb 27 03:47:43 2018 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Tue, 27 Feb 2018 09:47:43 +0100 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: <63e5b467-dabc-f24b-ed7c-717071e39beb@biologie.uni-freiburg.de> On 26.02.2018 15:41, Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. > Others have pointed out good solutions already, in particular, combining inspect and decorators or encapsulating the parameters in an object. Alternatively, you could reconsider your class design. Although I can't tell from your example whether this idea would be acceptable for your use case, consider removing your parameters from the class methods (except from __init__) altogether so the values passed during instantiation cannot be changed later. In exchange, add module-level functions corresponding to each of your class methods that accept **kwargs and that generate new instances of your class passing **kwargs on to __init__, then call the corresponding instance method. The stdlib textwrap module, for example, uses this approach. Best, Wolfgang From kirillbalunov at gmail.com Tue Feb 27 03:52:21 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Tue, 27 Feb 2018 11:52:21 +0300 Subject: Questions about `locals` builtin In-Reply-To: <87inaivs8x.fsf@handshake.de> References: <87inaivs8x.fsf@handshake.de> Message-ID: 2018-02-27 2:57 GMT+03:00 Terry Reedy : > The point of point 3 is that terminology and details would likely be > different if Python were freshly designed more or less as it is today, and > some things only make more or less sense in historical context. Learn what > you need to know to write code that works. > Thank you, I'm fairly familiar with the scope and namespace concepts in Python 3, and they are also very well described in the "Execution model" https://docs.python.org/3/reference/executionmodel.html#execution-model, for this special thanks to the person who wrote it ;-) I started using Python with ?Python 3.5 and I'm not familiar with the Python 2 features. But since Python 2 got into our discussion, I still have a question: a. Is this restriction for locals desirable in the implementation of CPython in Python 3? b. Or is it the result of temporary fixes for Python 2? Personally, I find the convenient functionality to update the local symbol table inside a function, similar to `globals`. Of course, I do not have the full picture and the difficulties with the flaws in implementing such functionality in CPython3. On the other hand, I understand that this additional dynamism as a whole may not be beneficial, and that local and global variables are determined at function compilation time. At this point, I only know one way to achieve this: `exec ('% s =% s'% (var_name, var_val))`, which I find clumsy enough. Funny, contradictory thoughts in my head :) Calling surrounding function local names collectively 'nonlocals' is the > result of months of bikeshedding. > Yes `nonlocals` is a cool fix in Python3. 2018-02-27 3:25 GMT+03:00 Steven D'Aprano : > Mostly because locals() predates MappingProxyType by many years, and also > because that restriction doesn't apply to other implementations of Python > such as Jython and IronPython. > > In CPython, the dict returned by locals() is a copy of the local > namespace, so modifying the dict doesn't modify the real local variables. > > (Well, sometimes it does, but not always. The story in Python 2 is really > complex.) Yes, I understand all the more the documentation specifies *New in version 3.3: class.MappingProxyType.* I'm not saying that this should be a standard for the Python language, especially in the context of what I wrote above. But the Python documentation already contains links to the features of the CPython implementation, (`id` for example). If the answer to the previous question is "Yes, such a restriction is desirable in CPython because, because, because ... and thereafter it is not planned to be changed." Then I do not see why it can not be made more explicit by changing `dict` to` types.MappingProxyType`. With the following change in the documentation: "Note The contents of this dictionary should be perceived as read-only mapping and should not be modified; The returned mapping type is implementation-dependent: changes may not affect the values of local variables used by the interpreter or the returned object may not support item assignment. CPython implementation detail: The returned object is `types.MappingProxyType` a read-only proxy of the current local symbol table." p.s.: Steven, this question was somewhat inspired by yours "Is there are good DRY fix for this painful design pattern?" With kind regards, -gdg From ben.usenet at bsb.me.uk Tue Feb 27 05:12:46 2018 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 27 Feb 2018 10:12:46 +0000 Subject: How to make Python run as fast (or faster) than Julia References: <8W2lC.362298$jQ2.204214@fx46.am4> Message-ID: <87woyypwpt.fsf@bsb.me.uk> Christian Gollwitzer writes: > George Marsaglia is a researcher who invented a couple of PRNGs which > are both simple (one of the first was called KISS) yet surprisingly > good. s/is/was/ Sadly, he died a few years ago (2011). -- Ben. From jenswaelkens at gmail.com Tue Feb 27 05:50:19 2018 From: jenswaelkens at gmail.com (jenswaelkens at gmail.com) Date: Tue, 27 Feb 2018 02:50:19 -0800 (PST) Subject: [newbie] how to remove empty lines from webpage/file Message-ID: Dear all, I try to get the numerical data from the following webpage: http://www.astro.oma.be/GENERAL/INFO/nzon/zon_2018.html With the following code-fragment I was already able to get a partial result: #!/usr/bin/env python #memo: install bs4 as follows: sudo easy_install bs4 # -*- coding: utf-8 -*- #3 lines below necessary to avoid encoding problem import sys reload(sys) sys.setdefaultencoding('utf8') import urllib2 file = open("testfile.txt","w") source = "http://www.astro.oma.be/GENERAL/INFO/nzon/zon_2018.html" page = urllib2.urlopen(source) from bs4 import BeautifulSoup soup = BeautifulSoup(page,'lxml') lines=soup.get_text() file.write(lines) file.close() I tried to delete the empty lines but I am totally stuck at this moment, can anyone help me further? thanks in advance jens From __peter__ at web.de Tue Feb 27 06:08:19 2018 From: __peter__ at web.de (Peter Otten) Date: Tue, 27 Feb 2018 12:08:19 +0100 Subject: matrix multiplication References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> <87muzvi4hs.fsf@otaria.sebmel.org> Message-ID: Seb wrote: > On Tue, 27 Feb 2018 12:25:30 +1300, > Gregory Ewing wrote: > >> Seb wrote: >>> I was wondering is whether there's a faster way of multiplying each >>> row (1x3) of a matrix by another matrix (3x3), compared to looping >>> through the matrix row by row as shown in the code. > >> Just multiply the two matrices together. > >> If A is an nx3 matrix and B is a 3x3 matrix, then C = A @ B is an nx3 >> matrix where C[i] = A[i] @ B. > >> (This is a property of matrix multiplication in general, nothing >> special about numpy.) > > I think that's only true if B is the same for every row in A. In the > code I posted, B varies by row of A. Yeah, you would have to substitute the N 3x3 matrices with an Nx3x3 tensor, though I don't know if numpy provides an op such that Nx3 op Nx3x3 --> desired result or op(Nx3, Nx3x3) --> desired result From bc at freeuk.com Tue Feb 27 06:27:21 2018 From: bc at freeuk.com (bartc) Date: Tue, 27 Feb 2018 11:27:21 +0000 Subject: How to make Python run as fast (or faster) than Julia In-Reply-To: References: <8W2lC.362298$jQ2.204214@fx46.am4> Message-ID: On 27/02/2018 02:27, Chris Angelico wrote: > On Tue, Feb 27, 2018 at 12:57 PM, bartc wrote: >> On 27/02/2018 00:35, Chris Angelico wrote: >> Anyway, even this pure Python version can deliver pseudo random numbers at >> some 200,000 per second, while the built-in generator does 450,000 per >> second, so it's not bad going. > > The built-in generator is using a completely different algorithm > though, so rate of generation isn't the entire story. How long is the > period of the one you're using? (How long before it loops?) I believe it's 5*2**1320480*(2**64-1) according to the author's comment. I haven't tested that. (By looping I understand that to mean, before the same sequence starts again. Because as the numbers are 64 bits, individual numbers will inevitably be repeated from time to time.) -- Bartc From ned at nedbatchelder.com Tue Feb 27 06:59:03 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 27 Feb 2018 06:59:03 -0500 Subject: Questions about `locals` builtin In-Reply-To: References: <87inaivs8x.fsf@handshake.de> Message-ID: <1322f99f-609b-2eac-48a9-9d9be5d08efc@nedbatchelder.com> On 2/27/18 3:52 AM, Kirill Balunov wrote: > a. Is this restriction for locals desirable in the implementation of > CPython in Python 3? > b. Or is it the result of temporary fixes for Python 2? My understanding is that the behavior of locals() is determined mostly by what is convenient for the implementors, so that they can keep regular code running as quickly as possible.? The answer to the question, "why can't we make locals() work more like I expect?" is, "because that would make things slower." > > Personally, I find the convenient functionality to update the local symbol > table inside a function, similar to `globals`. Can you show us an example of why you would want to update locals through locals()?? There might be more natural ways to solve your problem. --Ned. From steve+comp.lang.python at pearwood.info Tue Feb 27 08:59:59 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 13:59:59 +0000 (UTC) Subject: How to make Python run as fast (or faster) than Julia References: <8W2lC.362298$jQ2.204214@fx46.am4> Message-ID: On Tue, 27 Feb 2018 11:27:21 +0000, bartc wrote: [...] >> The built-in generator is using a completely different algorithm >> though, so rate of generation isn't the entire story. How long is the >> period of the one you're using? (How long before it loops?) > > I believe it's 5*2**1320480*(2**64-1) according to the author's comment. > > I haven't tested that. What, you haven't run a full period of the generator? I thought your code was fast! *wink* Sorry, I couldn't resist... :-) But for the record, if we could generate a million million numbers per second, that is one every picosecond, the total time would be of the order of 10**397504 years. That's a one followed by three hundred ninety- seven thousand zeroes. > (By looping I understand that to mean, before the same sequence starts > again. Because as the numbers are 64 bits, individual numbers will > inevitably be repeated from time to time.) Yes, that's what the period means: how long it takes for the entire sequence to repeat, not just a single number. -- Steve From ian.g.kelly at gmail.com Tue Feb 27 09:36:31 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 27 Feb 2018 07:36:31 -0700 Subject: matrix multiplication In-Reply-To: References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> <87muzvi4hs.fsf@otaria.sebmel.org> Message-ID: On Tue, Feb 27, 2018 at 4:08 AM, Peter Otten <__peter__ at web.de> wrote: > Seb wrote: > >> On Tue, 27 Feb 2018 12:25:30 +1300, >> Gregory Ewing wrote: >> >>> Seb wrote: >>>> I was wondering is whether there's a faster way of multiplying each >>>> row (1x3) of a matrix by another matrix (3x3), compared to looping >>>> through the matrix row by row as shown in the code. >> >>> Just multiply the two matrices together. >> >>> If A is an nx3 matrix and B is a 3x3 matrix, then C = A @ B is an nx3 >>> matrix where C[i] = A[i] @ B. >> >>> (This is a property of matrix multiplication in general, nothing >>> special about numpy.) >> >> I think that's only true if B is the same for every row in A. In the >> code I posted, B varies by row of A. > > Yeah, you would have to substitute the N 3x3 matrices with an Nx3x3 tensor, > though I don't know if numpy provides an op such that > > Nx3 op Nx3x3 --> desired result > > or > > op(Nx3, Nx3x3) --> desired result Nx1x3 @ Nx3x3 ought to do it, with the result being Nx1x3. From kirillbalunov at gmail.com Tue Feb 27 09:43:20 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Tue, 27 Feb 2018 17:43:20 +0300 Subject: Questions about `locals` builtin In-Reply-To: <1322f99f-609b-2eac-48a9-9d9be5d08efc@nedbatchelder.com> References: <87inaivs8x.fsf@handshake.de> <1322f99f-609b-2eac-48a9-9d9be5d08efc@nedbatchelder.com> Message-ID: 2018-02-27 14:59 GMT+03:00 Ned Batchelder : > On 2/27/18 3:52 AM, Kirill Balunov wrote: > >> a. Is this restriction for locals desirable in the implementation of >> CPython in Python 3? >> b. Or is it the result of temporary fixes for Python 2? >> > > My understanding is that the behavior of locals() is determined mostly by > what is convenient for the implementors, so that they can keep regular code > running as quickly as possible. The answer to the question, "why can't we > make locals() work more like I expect?" is, "because that would make things > slower." > Ok, but I in this case what is the benefit in Python 3.3+ in returning a copy of dict instead of MappingProxy? Personally, I find the convenient functionality to update the local symbol >> table inside a function, similar to `globals`. >> > > Can you show us an example of why you would want to update locals through > locals()? There might be more natural ways to solve your problem. > > The example from "Is there are good DRY fix for this painful design pattern?" https://mail.python.org/pipermail/python-list/2018- February/731218.html class Foo: def __init__(self, bashful, doc, dopey, grumpy, happy, sleepy, sneezy): self.bashful = bashful # etc def spam(self, bashful=None, doc=None, dopey=None, grumpy=None, happy=None, sleepy=None, sneezy=None): if bashful is None: bashful = self.bashful if doc is None: doc = self.doc if dopey is None: dopey = self.dopey if grumpy is None: grumpy = self.grumpy if happy is None: happy = self.happy if sleepy is None: sleepy = self.sleepy if sneezy is None: sneezy = self.sneezy # now do the real work... def eggs(self, bashful=None, # etc... ): if bashful is None: bashful = self.bashful # and so on and with the possibility to update `locals` the `spam` can be rewritten with: def spam(self, bashful=None, doc=None, dopey=None, grumpy=None, happy=None, sleepy=None, sneezy=None): loc = locals() for key, val in loc.items(): if val is None: loc[key] = getattr(self, key) In fact, I do not have a strict opinion on this matter. And I'd rather be glad that P?thon was less dynamic in some moments in favor of some optimizations. With kind regards, -gdg From kirillbalunov at gmail.com Tue Feb 27 09:55:15 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Tue, 27 Feb 2018 17:55:15 +0300 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: <63e5b467-dabc-f24b-ed7c-717071e39beb@biologie.uni-freiburg.de> References: <63e5b467-dabc-f24b-ed7c-717071e39beb@biologie.uni-freiburg.de> Message-ID: This validation can be also done with the use of annotations, while I find it super awful, I put this for one more example: from functools import wraps def validate(func): @wraps(func) def _wrap(self, *args, **kwargs): variables = func.__annotations__.keys() kwargs.update(zip(variables, args)) for var in variables - kwargs.keys(): kwargs[var] = getattr(self, var) return func(self, **kwargs) return _wrap class Foo: def __init__(self, bashful, doc, dopey, grumpy, happy, sleepy, sneezy): self.bashful = bashful self.doc = doc self.dopey = dopey self.grumpy = grumpy self.happy = happy self.sleepy = sleepy self.sneezy = sneezy @validate def spam(self, bashful:'Any'=None, doc:'Any'=None, dopey:'Any'=None, grumpy:'Any'=None, happy:'Any'=None, sleepy:'Any'=None, sneezy:'Any'=None): return bashful, doc, dopey, grumpy, happy, sleepy, sneezy a = Foo(1,2,3,4,5,6,7) a.spam(grumpy='Hello') With kind regards, -gdg From drsalists at gmail.com Tue Feb 27 10:25:53 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 27 Feb 2018 07:25:53 -0800 Subject: [newbie] how to remove empty lines from webpage/file In-Reply-To: References: Message-ID: Perhaps replace: lines=soup.get_text() file.write(lines) ...with something like: text = soup.get_text() lines = text.split('\n') for line in lines: if line.strip(): file.write('%s\n' % (line, )) (untested) On Tue, Feb 27, 2018 at 2:50 AM, wrote: > Dear all, > I try to get the numerical data from the following webpage: > http://www.astro.oma.be/GENERAL/INFO/nzon/zon_2018.html > > With the following code-fragment I was already able to get a partial result: > > #!/usr/bin/env python > #memo: install bs4 as follows: sudo easy_install bs4 > # -*- coding: utf-8 -*- > #3 lines below necessary to avoid encoding problem > import sys > reload(sys) > sys.setdefaultencoding('utf8') > import urllib2 > file = open("testfile.txt","w") > source = "http://www.astro.oma.be/GENERAL/INFO/nzon/zon_2018.html" > page = urllib2.urlopen(source) > from bs4 import BeautifulSoup > soup = BeautifulSoup(page,'lxml') > lines=soup.get_text() > file.write(lines) > file.close() > > I tried to delete the empty lines but I am totally stuck at this moment, can anyone help me further? > > thanks in advance > jens > -- > https://mail.python.org/mailman/listinfo/python-list From kirillbalunov at gmail.com Tue Feb 27 10:45:43 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Tue, 27 Feb 2018 18:45:43 +0300 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: <63e5b467-dabc-f24b-ed7c-717071e39beb@biologie.uni-freiburg.de> Message-ID: Of course you can do the same without annotations, but with the introduction of private attribute while your API it is under active development: from functools import wraps def validate(func): @wraps(func) def _wrap(self, *args, **kwargs): variables = self._vars # Here kwargs.update(zip(variables, args)) for var in variables - kwargs.keys(): kwargs[var] = getattr(self, var) return func(self, **kwargs) return _wrap class Foo: def __init__(self, bashful, doc, dopey, grumpy, happy, sleepy, sneezy): self.bashful = bashful self.doc = doc self.dopey = dopey self.grumpy = grumpy self.happy = happy self.sleepy = sleepy self.sneezy = sneezy self._vars = set(v for v in self.__dict__ if not v.startswith('_')) # should be deleted when finish @validate def spam(self, bashful=None, doc=None, dopey=None, grumpy=None, happy=None, sleepy=None, sneezy=None): return bashful, doc, dopey, grumpy, happy, sleepy, sneezy With kind regards, -gdg From spluque at gmail.com Tue Feb 27 11:02:54 2018 From: spluque at gmail.com (Seb) Date: Tue, 27 Feb 2018 10:02:54 -0600 Subject: matrix multiplication References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> <87muzvi4hs.fsf@otaria.sebmel.org> Message-ID: <87lgfeju8h.fsf@otaria.sebmel.org> On Tue, 27 Feb 2018 07:36:31 -0700, Ian Kelly wrote: > On Tue, Feb 27, 2018 at 4:08 AM, Peter Otten <__peter__ at web.de> wrote: >> Seb wrote: >>> On Tue, 27 Feb 2018 12:25:30 +1300, >>> Gregory Ewing wrote: >>>> Seb wrote: >>>>> I was wondering is whether there's a faster way of multiplying >>>>> each row (1x3) of a matrix by another matrix (3x3), compared to >>>>> looping through the matrix row by row as shown in the code. >>>> Just multiply the two matrices together. >>>> If A is an nx3 matrix and B is a 3x3 matrix, then C = A @ B is an >>>> nx3 matrix where C[i] = A[i] @ B. >>>> (This is a property of matrix multiplication in general, nothing >>>> special about numpy.) >>> I think that's only true if B is the same for every row in A. In >>> the code I posted, B varies by row of A. >> Yeah, you would have to substitute the N 3x3 matrices with an Nx3x3 >> tensor, though I don't know if numpy provides an op such that >> Nx3 op Nx3x3 --> desired result >> or >> op(Nx3, Nx3x3) --> desired result > Nx1x3 @ Nx3x3 ought to do it, with the result being Nx1x3. That's right. I just tried this manipulation by replacing the last block of code in my example, from the line above `for` loop with: ---<--------------------cut here---------------start------------------->--- # Alternative using `np.matmul` uvw_alt = uvw.reshape((uvw.shape[0], 1, uvw.shape[1])) bmats = np.asarray(map(randint_mat, maxint)) uvw_rots_alt = np.matmul(uvw_alt, bmats).squeeze() ---<--------------------cut here---------------end--------------------->--- Interestingly, the time savings from IPython are not spectacular: %run -t -N100 loop_approach.py IPython CPU timings (estimated): Total runs performed: 100 Times : Total Per run User : 0.28 s, 0.00 s. System : 0.00 s, 0.00 s. Wall time: 0.28 s. %run -t -N100 matmul_approach.py IPython CPU timings (estimated): Total runs performed: 100 Times : Total Per run User : 0.17 s, 0.00 s. System : 0.00 s, 0.00 s. Wall time: 0.18 s. -- Seb From alister.ware at ntlworld.com Tue Feb 27 11:36:04 2018 From: alister.ware at ntlworld.com (alister) Date: Tue, 27 Feb 2018 16:36:04 GMT Subject: help me ? References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On Mon, 26 Feb 2018 01:40:16 -0800, sotaro237 wrote: > Define 2 lists. The first one must contain the integer values 1, 2 and 3 > and the second one the string values a, b and c. Iterate through both > lists to create another list that contains all the combinations of the A > and B elements. The final list should look like one of the 2 lists: > 1. [1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c] > 2. [a1, a2. a3, b1, b2, b3, c1, c2, c3] > BONUS: Make the final list contain all possible combinations : [1a, a1, > 1b, b1, 1c, c1, 2a, a2, 2b, b2, 2c, c2, 3a, a3, 3b, b3, 3c, c3] > > Help me ! Try staying awake in class where I am sure what you need would have been covered before the assignment was set failing that try reading any of the python tutorials available on the internet -- A career is great, but you can't run your fingers through its hair. From nomail at adres.net Tue Feb 27 11:54:35 2018 From: nomail at adres.net (Sir Real) Date: Tue, 27 Feb 2018 10:54:35 -0600 Subject: help me ? References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On Mon, 26 Feb 2018 01:40:16 -0800 (PST), sotaro237 at gmail.com wrote: >Define 2 lists. The first one must contain the integer values 1, 2 and 3 and the second one the string values a, b and c. Iterate through both lists to create another list that contains all the combinations of the A and B elements. The final list should look like one of the 2 lists: >1. [1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c] >2. [a1, a2. a3, b1, b2, b3, c1, c2, c3] >BONUS: Make the final list contain all possible combinations : [1a, a1, 1b, b1, 1c, c1, 2a, a2, 2b, b2, 2c, c2, 3a, a3, 3b, b3, 3c, c3] > >Help me ! #a list of integers nums = [1, 2, 3] #a list of letters ltrs = ['a', 'b', 'c'] #a list to hold the result rslt = [] for i in nums: for j in ltrs: rslt.append(str(i) + j) #for bonus points uncomment the following line #rslt.append(j + str(i)) print(rslt) '''RESULT ['1a', '1b', '1c', '2a', '2b', '2c', '3a', '3b', '3c'] BONUS RESULT ['1a', 'a1', '1b', 'b1', '1c', 'c1', '2a', 'a2', '2b', 'b2', '2c', 'c2', '3a', 'a3', '3b', 'b3', '3c', 'c3'] ''' From ikorot01 at gmail.com Tue Feb 27 12:16:29 2018 From: ikorot01 at gmail.com (Igor Korot) Date: Tue, 27 Feb 2018 11:16:29 -0600 Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: Hi, On Tue, Feb 27, 2018 at 10:54 AM, Sir Real wrote: > On Mon, 26 Feb 2018 01:40:16 -0800 (PST), sotaro237 at gmail.com wrote: > >>Define 2 lists. The first one must contain the integer values 1, 2 and 3 and the second one the string values a, b and c. Iterate through both lists to create another list that contains all the combinations of the A and B elements. The final list should look like one of the 2 lists: >>1. [1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c] >>2. [a1, a2. a3, b1, b2, b3, c1, c2, c3] >>BONUS: Make the final list contain all possible combinations : [1a, a1, 1b, b1, 1c, c1, 2a, a2, 2b, b2, 2c, c2, 3a, a3, 3b, b3, 3c, c3] >> >>Help me ! > > #a list of integers > nums = [1, 2, 3] > > #a list of letters > ltrs = ['a', 'b', 'c'] > > #a list to hold the result > rslt = [] > > for i in nums: > for j in ltrs: > rslt.append(str(i) + j) > #for bonus points uncomment the following line > #rslt.append(j + str(i)) > > print(rslt) > > > '''RESULT > ['1a', '1b', '1c', > '2a', '2b', '2c', > '3a', '3b', '3c'] > > BONUS RESULT > ['1a', 'a1', '1b', 'b1', '1c', 'c1', > '2a', 'a2', '2b', 'b2', '2c', 'c2', > '3a', 'a3', '3b', 'b3', '3c', 'c3'] Congratulations! You have an "A" for solving the problem and "F" for helping the guy cheat. You should be expelled from the course. ;-) Thank you. > ''' > -- > https://mail.python.org/mailman/listinfo/python-list From python.list at tim.thechases.com Tue Feb 27 12:47:13 2018 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 27 Feb 2018 11:47:13 -0600 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: <20180227114713.6582f82c@bigbox.christie.dr> Something like the following might do the trick. As an added benefit, it's easy to set all the defaults automatically in __init__ as well without hand-adding "self.dopey = dopey". On the down side, in the non-__init__ functions, you have to use kwargs["dopey"] and the like. It also involves tacking an "__init_args" onto your object so that the decorator knows what was passed to the __init__ function. -tkc from functools import wraps from inspect import getargspec def template(original_init_fn): args, varargs, keywords, defaults = getargspec(original_init_fn) assert varargs is keywords is None arg_dict = dict(zip(args[-len(defaults):], defaults)) @wraps(original_init_fn) def new_init_fn(self, *args, **kwargs): self.__init_args = arg_dict.copy() self.__init_args.update(kwargs) # if you don't want to automatically set attributes # remove these next two lines for k, v in self.__init_args.items(): setattr(self, k, v) return original_init_fn(self, *args, **kwargs) def templatify(fn): @wraps(fn) def new_templated_fn(self, *args, **kwargs): for k, v in self.__init_args.items(): if k not in kwargs: kwargs[k] = v return fn(self, *args, **kwargs) return new_templated_fn new_init_fn.templatify = templatify return new_init_fn class Foo: @template def __init__(self, bashful=None, dopey=None, doc="On definition", ): pass # look, ma, no manual assignment! @__init__.templatify def myfunc(self, **kwargs): print(kwargs) f1 = Foo() f2 = Foo(bashful="on init", dopey="on init") for fn in (f1, f2): fn.myfunc() fn.myfunc(bashful="on myfunc") On 2018-02-26 14:41, Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) > assigned in `__init__`. The class then has a number of methods > which accept *optional* arguments with the same names as the > constructor/initialiser parameters. If those arguments are None, > the defaults are taken from the instance attributes. > > An example might be something like this: > > > class Foo: > def __init__(self, bashful, doc, dopey, grumpy, > happy, sleepy, sneezy): > self.bashful = bashful # etc > > def spam(self, bashful=None, doc=None, dopey=None, > grumpy=None, happy=None, sleepy=None, > sneezy=None): > if bashful is None: > bashful = self.bashful > if doc is None: > doc = self.doc > if dopey is None: > dopey = self.dopey > if grumpy is None: > grumpy = self.grumpy > if happy is None: > happy = self.happy > if sleepy is None: > sleepy = self.sleepy > if sneezy is None: > sneezy = self.sneezy > # now do the real work... > > def eggs(self, bashful=None, # etc... > ): > if bashful is None: > bashful = self.bashful > # and so on > > > There's a lot of tedious boilerplate repetition in this, and to add > insult to injury the class is still under active development with > an unstable API, so every time I change one of the parameters, or > add a new one, I have to change it in over a dozen places. > > Is there a good fix for this to reduce the amount of boilerplate? > > > Thanks, > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list From ian.g.kelly at gmail.com Tue Feb 27 12:47:15 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 27 Feb 2018 10:47:15 -0700 Subject: matrix multiplication In-Reply-To: <87lgfeju8h.fsf@otaria.sebmel.org> References: <87lgfhnesw.fsf@otaria.sebmel.org> <87efl7pu8t.fsf@gmail.com> <87muzvi4hs.fsf@otaria.sebmel.org> <87lgfeju8h.fsf@otaria.sebmel.org> Message-ID: On Tue, Feb 27, 2018 at 9:02 AM, Seb wrote: > That's right. I just tried this manipulation by replacing the last > block of code in my example, from the line above `for` loop with: > > ---<--------------------cut here---------------start------------------->--- > # Alternative using `np.matmul` > uvw_alt = uvw.reshape((uvw.shape[0], 1, uvw.shape[1])) > bmats = np.asarray(map(randint_mat, maxint)) > uvw_rots_alt = np.matmul(uvw_alt, bmats).squeeze() > ---<--------------------cut here---------------end--------------------->--- > > Interestingly, the time savings from IPython are not spectacular: The second line is still calling the Python function randint_mat 1000 times. That should be possible to do without the Python function using broadcasting, if https://github.com/numpy/numpy/issues/6745 were fixed. Otherwise, I don't know of a way to vectorize that part efficiently. From ian.g.kelly at gmail.com Tue Feb 27 12:56:18 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 27 Feb 2018 10:56:18 -0700 Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On Tue, Feb 27, 2018 at 10:16 AM, Igor Korot wrote: > Congratulations! > You have an "A" for solving the problem and "F" for helping the guy cheat. > You should be expelled from the course. In my experience, this is what happens pretty much every time. Somebody posts a homework question asking for the answer, a few people say something to the effect of, "This looks like homework. What have you tried so far?" Then some other bozo comes along who just likes solving easy problems and hands up the answer on a silver platter. Cheaters are gonna cheat. In the unlikely event they don't get the answer here, they'll probably just manage to convince somebody to do the work for them somewhere else. Honestly, I don't know if it's even worth the bother to engage. From larry.martell at gmail.com Tue Feb 27 13:14:51 2018 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 27 Feb 2018 13:14:51 -0500 Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On Tue, Feb 27, 2018 at 12:56 PM, Ian Kelly wrote: > On Tue, Feb 27, 2018 at 10:16 AM, Igor Korot wrote: >> Congratulations! >> You have an "A" for solving the problem and "F" for helping the guy cheat. >> You should be expelled from the course. > > In my experience, this is what happens pretty much every time. > Somebody posts a homework question asking for the answer, a few people > say something to the effect of, "This looks like homework. What have > you tried so far?" Then some other bozo comes along who just likes > solving easy problems and hands up the answer on a silver platter. > > Cheaters are gonna cheat. In the unlikely event they don't get the > answer here, they'll probably just manage to convince somebody to do > the work for them somewhere else. Honestly, I don't know if it's even > worth the bother to engage. When I was in college back in the 70's they made people in majors like printing or chemistry, for example, take 1 programming course. They were always clueless and I often wrote programs for them - my typical fee was a case of beer. From Bischoop at Free.net Tue Feb 27 13:38:44 2018 From: Bischoop at Free.net (Ziggy) Date: Tue, 27 Feb 2018 18:38:44 GMT Subject: help me ? References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On 2018-02-26, sotaro237 at gmail.com wrote: > > Help me ! a=[1,2,3,] b=["a","b","c"] x=[] z=[] bonus=[] for digits in a: for letters in b: x.append(str(digits) + letters) bonus.append(letters + str(digits)) for letter in b: for number in a: z.append(letter + str(number)) bonus.append(str(number) + letter) print ("1:",x) print ("2:",z) print("Bonus:",bonus) From grant.b.edwards at gmail.com Tue Feb 27 13:54:12 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 27 Feb 2018 18:54:12 +0000 (UTC) Subject: help me ? References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On 2018-02-27, Ian Kelly wrote: > On Tue, Feb 27, 2018 at 10:16 AM, Igor Korot wrote: >> Congratulations! >> You have an "A" for solving the problem and "F" for helping the guy cheat. >> You should be expelled from the course. > > In my experience, this is what happens pretty much every time. > Somebody posts a homework question asking for the answer, a few people > say something to the effect of, "This looks like homework. What have > you tried so far?" Then some other bozo comes along who just likes > solving easy problems and hands up the answer on a silver platter. If the student is actively trying to avoid learning something, there's nothing you can do to help them. They're just wasting their own time and money. The fun part is giving them a solution that's so obscure and "clever" that it technically meets the stated requirement but is so far from what the instructor wanted that they don't get credit for it (and there's no way the student will be able explain how it works to the instructor). -- Grant Edwards grant.b.edwards Yow! We just joined the at civil hair patrol! gmail.com From jugurtha.hadjar at gmail.com Tue Feb 27 13:56:50 2018 From: jugurtha.hadjar at gmail.com (Jugurtha Hadjar) Date: Tue, 27 Feb 2018 19:56:50 +0100 Subject: Is there are good DRY fix for this painful design pattern? In-Reply-To: References: Message-ID: <0e649432-a775-2da2-4247-ad423d21b78c@gmail.com> On 02/26/2018 03:41 PM, Steven D'Aprano wrote: > I have a class with a large number of parameters (about ten) assigned in > `__init__`. The class then has a number of methods which accept > *optional* arguments with the same names as the constructor/initialiser > parameters. If those arguments are None, the defaults are taken from the > instance attributes. > > An example might be something like this: > > > class Foo: > def __init__(self, bashful, doc, dopey, grumpy, > happy, sleepy, sneezy): > self.bashful = bashful # etc > > def spam(self, bashful=None, doc=None, dopey=None, > grumpy=None, happy=None, sleepy=None, > sneezy=None): > if bashful is None: > bashful = self.bashful > if doc is None: > doc = self.doc > if dopey is None: > dopey = self.dopey > if grumpy is None: > grumpy = self.grumpy > if happy is None: > happy = self.happy > if sleepy is None: > sleepy = self.sleepy > if sneezy is None: > sneezy = self.sneezy > # now do the real work... > > def eggs(self, bashful=None, # etc... > ): > if bashful is None: > bashful = self.bashful > # and so on > > > There's a lot of tedious boilerplate repetition in this, and to add > insult to injury the class is still under active development with an > unstable API, so every time I change one of the parameters, or add a new > one, I have to change it in over a dozen places. > > Is there a good fix for this to reduce the amount of boilerplate? > > > Thanks, > > > def snatchit(func): ??? def snatched(self, *args, **kwargs): ??????? frame = inspect.signature(func) ??????? for name in frame.parameters: ??????????? if name != 'self': ??????????????? kwargs.setdefault(name, getattr(self, name)) ??????? return func(self, *args, **kwargs) ??? snatched.__name__ = func.__name__ ??? snatched.__doc__ = func.__doc__ ??? return snatched class Foo(object): ??? def __init__(self, sleepy, groggy): ??????? self.sleepy = sleepy ??????? self.groggy = groggy ??? @snatchit ??? def spam(self, sleepy=None): ??????? print("I am spam. sleepy=", sleepy) ??? @snatchit ??? def ham(self, sleepy=None, groggy=None): ??????? print("I am ham. sleepy=", sleepy, " and groggy=", groggy) >>> f = Foo(1, 19) >>> f.spam() I am spam. sleepy= 1 >>> f.spam(sleepy=8) I am spam. sleepy= 8 >>> f.ham(sleepy=8) I am ham. sleepy= 8? and groggy= 19 >>> f.ham(sleepy=17, groggy=888) I am ham. sleepy= 17? and groggy= 888 -- ~ Jugurtha Hadjar, From phd at phdru.name Tue Feb 27 16:20:08 2018 From: phd at phdru.name (Oleg Broytman) Date: Tue, 27 Feb 2018 22:20:08 +0100 Subject: Cheetah 3.0.1 Message-ID: <20180227212008.mlz57d7kgcxc6mab@phdru.name> Hello! I'm pleased to announce version 3.0.1, the first bugfix release of branch 3.0 of CheetahTemplate3. What's new in CheetahTemplate3 ============================== Bug fixes: - Fix a minor bug in Compiler. What is CheetahTemplate3 ======================== Cheetah3 is a free and open source template engine. It's a fork of the original CheetahTemplate library. Python 2.7 or 3.3+ is required. Where is CheetahTemplate3 ========================= Site: http://cheetahtemplate.org/ Development: https://github.com/CheetahTemplate3 Download: https://pypi.python.org/pypi/Cheetah3/3.0.1 News and changes: http://cheetahtemplate.org/news.html StackOverflow: https://stackoverflow.com/questions/tagged/cheetah Example ======= Below is a simple example of some Cheetah code, as you can see it's practically Python. You can import, inherit and define methods just like in a regular Python module, since that's what your Cheetah templates are compiled to :) :: #from Cheetah.Template import Template #extends Template #set $people = [{'name' : 'Tom', 'mood' : 'Happy'}, {'name' : 'Dick', 'mood' : 'Sad'}, {'name' : 'Harry', 'mood' : 'Hairy' How are you feeling?
    #for $person in $people
  • $person['name'] is $person['mood']
    • #end for
Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From gbs.deadeye at gmail.com Tue Feb 27 16:50:14 2018 From: gbs.deadeye at gmail.com (=?UTF-8?Q?Andre_M=C3=BCller?=) Date: Tue, 27 Feb 2018 21:50:14 +0000 Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: Hello, it's a duplicate: https://python-forum.io/Thread-Working-with-lists-homework-2 I have seen this more than one time. We don't like it. You keep people busy with one question at different places. You need two lists and one empty list. One outer loop iterating over the first list and one inner loop iterating over the second list. In the inner loop you concatenate the two elements from the outer-loop and inner-loop. Then you append them to the empty list. This text is 10 times longer as the source code... Complicated solution: from string import ascii_lowercase as letter list1 = [str(i) + c for i in range(1,4) for c in letter[:3]] list2 = [c[::-1] for c in list1] But this won't help you. Before you understand the code above, you have to understand for-loops and nested for-loops. Then you can proceed with list comprehensions. But I don't see that your intention is to learn and understand Python. You just want to finish your homework, which bother you. Greetings Andre From ian.g.kelly at gmail.com Tue Feb 27 17:52:37 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 27 Feb 2018 15:52:37 -0700 Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On Tue, Feb 27, 2018 at 2:50 PM, Andre M?ller wrote: > Hello, > > it's a duplicate: > https://python-forum.io/Thread-Working-with-lists-homework-2 > > I have seen this more than one time. We don't like it. You keep people busy > with one question at different places. You assume that it was posted by the same person in both places, and not by two different students in the same class. From steve+comp.lang.python at pearwood.info Tue Feb 27 18:51:33 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 27 Feb 2018 23:51:33 +0000 (UTC) Subject: help me ? References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On Tue, 27 Feb 2018 10:56:18 -0700, Ian Kelly wrote: > Cheaters are gonna cheat. In the unlikely event they don't get the > answer here, they'll probably just manage to convince somebody to do the > work for them somewhere else. Honestly, I don't know if it's even worth > the bother to engage. Its worth the bother for the 5% who aren't *intentionally* cheating, but don't realise it, and just need a reminder that they need to do the work themselves. (Or at least give co-credit to those they got help from.) Or those who aren't actually doing homework, but are engaged in self- learning. There has to be some penalty for cheating, even if its only the minuscule amount of social disapproval that comes from random strangers on the internet telling you off, or we'll be even more overloaded by cheaters than we already are. -- Steve From michael.stemper at gmail.com Tue Feb 27 18:55:01 2018 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 27 Feb 2018 17:55:01 -0600 Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> <0001HW.20443BD7002FEE45150E102CF@news.individual.net> Message-ID: On 2018-02-26 07:17, Stefan Ram wrote: > Percival John Hackworth quoted: >>> Define 2 lists. The first one must contain the integer values 1, 2 and 3 > > a =[ 1, 2, 3 ] > >>> and the second one the string values a, b and c. > > b =[ 'a', 'b', 'c'] > >>> Iterate through both lists to create another list that >>> contains all the combinations of the A and B > > for i in a: > result = [ '1a', '1b', '1c', '2a', '2b', '2c', '3a', '3b', '3c' ] > > for j in b: > result = [ '1a', '1b', '1c', '2a', '2b', '2c', '3a', '3b', '3c' ] That's absolutely wonderful! However, I'd like to suggest one change, which would allow greater CPU utilization: for i in a: for j in b: # Sorry, I'm not PEP-8 compliant result = [ '1a', '1b', '1c', '2a', '2b', '2c', '3a', '3b', '3c' ] -- Michael F. Stemper Always remember that you are unique. Just like everyone else. From larry.martell at gmail.com Tue Feb 27 19:29:50 2018 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 27 Feb 2018 19:29:50 -0500 Subject: psutil Message-ID: Trying to install psutil (with pip install psutil) on Red Hat EL 7. It's failing with: Python.h: No such file or directory Typically that means the python devel libs are not installed, but they are: [root at liszt ~]# yum install python-devel Package python-devel-2.7.5-58.el7.x86_64 already installed and latest version Nothing to do Anyone have any idea what to try next? From chema at rinzewind.org Tue Feb 27 19:36:27 2018 From: chema at rinzewind.org (=?iso-8859-1?Q?Jos=E9_Mar=EDa?= Mateos) Date: Tue, 27 Feb 2018 19:36:27 -0500 Subject: psutil In-Reply-To: References: Message-ID: <20180228003627.GB6492@miniequipaje> On Tue, Feb 27, 2018 at 07:29:50PM -0500, Larry Martell wrote: > Trying to install psutil (with pip install psutil) on Red Hat EL 7. > It's failing with: > > Python.h: No such file or directory Two questions come to my mind: - Does it work if you try to install some other package? - Is `pip` by any change trying to install a Python 3 package, but you only have the libraries for Python 2 installed? Cheers, -- Jos? Mar?a Mateos https://rinzewind.org/blog-es || https://rinzewind.org/blog-en From rosuav at gmail.com Tue Feb 27 19:37:02 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Feb 2018 11:37:02 +1100 Subject: psutil In-Reply-To: References: Message-ID: On Wed, Feb 28, 2018 at 11:29 AM, Larry Martell wrote: > Trying to install psutil (with pip install psutil) on Red Hat EL 7. > It's failing with: > > Python.h: No such file or directory > > Typically that means the python devel libs are not installed, but they are: > > [root at liszt ~]# yum install python-devel > Package python-devel-2.7.5-58.el7.x86_64 already installed and latest version > Nothing to do > > Anyone have any idea what to try next? Are you trying to install that into Python 3 or Python 2? Might need to grab python3-devel. ChrisA From larry.martell at gmail.com Tue Feb 27 19:47:04 2018 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 27 Feb 2018 19:47:04 -0500 Subject: psutil In-Reply-To: References: Message-ID: On Tue, Feb 27, 2018 at 7:37 PM, Chris Angelico wrote: > On Wed, Feb 28, 2018 at 11:29 AM, Larry Martell wrote: >> Trying to install psutil (with pip install psutil) on Red Hat EL 7. >> It's failing with: >> >> Python.h: No such file or directory >> >> Typically that means the python devel libs are not installed, but they are: >> >> [root at liszt ~]# yum install python-devel >> Package python-devel-2.7.5-58.el7.x86_64 already installed and latest version >> Nothing to do >> >> Anyone have any idea what to try next? > > Are you trying to install that into Python 3 or Python 2? Might need > to grab python3-devel. Python 2 From larry.martell at gmail.com Tue Feb 27 19:49:43 2018 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 27 Feb 2018 19:49:43 -0500 Subject: psutil In-Reply-To: <20180228003627.GB6492@miniequipaje> References: <20180228003627.GB6492@miniequipaje> Message-ID: On Tue, Feb 27, 2018 at 7:36 PM, Jos? Mar?a Mateos wrote: > On Tue, Feb 27, 2018 at 07:29:50PM -0500, Larry Martell wrote: >> Trying to install psutil (with pip install psutil) on Red Hat EL 7. >> It's failing with: >> >> Python.h: No such file or directory > > Two questions come to my mind: > > - Does it work if you try to install some other package? Yes, I have installed other packages (e.g. numpy) > - Is `pip` by any change trying to install a Python 3 package, but you > only have the libraries for Python 2 installed? No, here is the gcc command line that failed: gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/opt/rh/python27/root/usr/include/python2.7 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_common.o From m at funkyhat.org Tue Feb 27 20:30:51 2018 From: m at funkyhat.org (Matt Wheeler) Date: Wed, 28 Feb 2018 01:30:51 +0000 (UTC) Subject: psutil In-Reply-To: References: <20180228003627.GB6492@miniequipaje> Message-ID: On Wed, 28 Feb 2018, 00:49 Larry Martell, wrote: > On Tue, Feb 27, 2018 at 7:36 PM, Jos? Mar?a Mateos > wrote: > > On Tue, Feb 27, 2018 at 07:29:50PM -0500, Larry Martell wrote: > >> Trying to install psutil (with pip install psutil) on Red Hat EL 7. > >> It's failing with: > >> > >> Python.h: No such file or directory > > > > Two questions come to my mind: > > > > - Does it work if you try to install some other package? > > Yes, I have installed other packages (e.g. numpy) > Did that install definitely involve a build from source? > - Is `pip` by any change trying to install a Python 3 package, but you > > only have the libraries for Python 2 installed? > > No, here is the gcc command line that failed: > > gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 > -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 > -I/opt/rh/python27/root/usr/include/python2.7 -c > psutil/_psutil_common.c -o > build/temp.linux-x86_64-2.7/psutil/_psutil_common.o > Based on the include path you have here this looks like you have a non standard python package for EL7 (in addition to the system one, or you'd be having a worse day), which is probably the root of your problem. Are you using python packages from scl perhaps? If so you probably need to install python27-python-devel (found at http://mirror.centos.org/centos/7/sclo/x86_64/rh/python27/) > -- -- Matt Wheeler http://funkyh.at From amber.of.luxor at gmail.com Tue Feb 27 21:46:25 2018 From: amber.of.luxor at gmail.com (amber) Date: Wed, 28 Feb 2018 02:46:25 +0000 Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: <609b2105-6384-f020-f1e8-78aef50c4952@gmail.com> On 02/27/2018 06:54 PM, Grant Edwards wrote: > The fun part is giving them a solution that's so obscure and "clever" > that it technically meets the stated requirement but is so far from > what the instructor wanted that they don't get credit for it (and > there's no way the student will be able explain how it works to the > instructor). Isn't that the point of doing their homework. Amber From best_lay at yahoo.com Tue Feb 27 22:15:31 2018 From: best_lay at yahoo.com (Wildman) Date: Tue, 27 Feb 2018 21:15:31 -0600 Subject: psutil References: Message-ID: On Tue, 27 Feb 2018 19:29:50 -0500, Larry Martell wrote: > Trying to install psutil (with pip install psutil) on Red Hat EL 7. > It's failing with: > > Python.h: No such file or directory > > Typically that means the python devel libs are not installed, but they are: > > [root at liszt ~]# yum install python-devel > Package python-devel-2.7.5-58.el7.x86_64 already installed and latest version > Nothing to do > > Anyone have any idea what to try next? I am not familiar with Red Hat and the RPM package system but on my Debian based system it is in the repository as an installable package. The package names are python-psutil and python3-psutil. -- GNU/Linux user #557453 The cow died so I don't need your bull! From jladasky at itu.edu Tue Feb 27 22:43:32 2018 From: jladasky at itu.edu (jladasky at itu.edu) Date: Tue, 27 Feb 2018 19:43:32 -0800 (PST) Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: <04591b86-19e7-46c8-9ffc-0b3980769213@googlegroups.com> On Tuesday, February 27, 2018 at 10:56:52 AM UTC-8, Grant Edwards wrote: > If the student is actively trying to avoid learning something, there's > nothing you can do to help them. They're just wasting their own time > and money. This is part of the reason why interviews for software developer jobs have gotten so crazy. We have to weed out the impostors after they graduate. > The fun part is giving them a solution that's so obscure and "clever" > that it technically meets the stated requirement but is so far from > what the instructor wanted that they don't get credit for it (and > there's no way the student will be able explain how it works to the > instructor). It doesn't even need to be an obscure solution, it just needs to make use of things you don't expect a novice programmer to use, like lambdas. A recent example: https://groups.google.com/forum/#!original/comp.lang.python/0gYm2g3BA2A/s9xbBG1GAwAJ From rantingrickjohnson at gmail.com Tue Feb 27 23:37:38 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 27 Feb 2018 20:37:38 -0800 (PST) Subject: help me ? In-Reply-To: References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> Message-ID: On Tuesday, February 27, 2018 at 12:56:52 PM UTC-6, Grant Edwards wrote: [...] > The fun part is giving them a solution that's so obscure and "clever" > that it technically meets the stated requirement but is so far from > what the instructor wanted that they don't get credit for it (and > there's no way the student will be able explain how it works to the > instructor). Even for the unobscure and unclever solutions (like the one Sir Real offered), i doubt the cheaters could explain them to any level of expertise that would be acceptable to a wise instructor. @OP: If after submitting your copy/paste solution the instructor calls you over for a chat, it's okay to be nervous, because typically this means you're about to be outed and made a complete fool of in front of the whole class. Yep, teachers are sadistic like that. Have a nice day! From nad at python.org Wed Feb 28 01:04:32 2018 From: nad at python.org (Ned Deily) Date: Wed, 28 Feb 2018 01:04:32 -0500 Subject: [RELEASE] Python 3.7.0b2 is now available for testing Message-ID: <66EE6E10-2815-4E53-841C-A8896BFA48A5@python.org> On behalf of the Python development community and the Python 3.7 release team, I'm happy to announce the availability of Python 3.7.0b2. b2 is the second of four planned beta releases of Python 3.7, the next major release of Python, and marks the end of the feature development phase for 3.7. You can find Python 3.7.0b2 here: https://www.python.org/downloads/release/python-370b2/ Among the new major new features in Python 3.7 are: * PEP 538, Coercing the legacy C locale to a UTF-8 based locale * PEP 539, A New C-API for Thread-Local Storage in CPython * PEP 540, UTF-8 mode * PEP 552, Deterministic pyc * PEP 553, Built-in breakpoint() * PEP 557, Data Classes * PEP 560, Core support for typing module and generic types * PEP 562, Module __getattr__ and __dir__ * PEP 563, Postponed Evaluation of Annotations * PEP 564, Time functions with nanosecond resolution * PEP 565, Show DeprecationWarning in __main__ * PEP 567, Context Variables Please see "What?s New In Python 3.7" for more information. Additional documentation for these features and for other changes will be provided during the beta phase. https://docs.python.org/3.7/whatsnew/3.7.html Beta releases are intended to give you the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. We strongly encourage you to test your projects with 3.7 during the beta phase and report issues found to https://bugs.python.org as soon as possible. While the release is feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2018-05-21). Our goal is have no ABI changes after beta 3 and no code changes after rc1. To achieve that, it will be extremely important to get as much exposure for 3.7 as possible during the beta phase. Attention macOS users: as of b1, there is a new installer variant for macOS 10.9+ that includes a built-in version of Tcl/Tk 8.6. This variant is expected to become the default version when 3.7.0 releases. Check it out! We welcome your feedback. Please keep in mind that this is a preview release and its use is not recommended for production environments. The next planned release of Python 3.7 will be 3.7.0b3, currently scheduled for 2018-03-26. More information about the release schedule can be found here: https://www.python.org/dev/peps/pep-0537/ -- Ned Deily nad at python.org -- [] From dieter at handshake.de Wed Feb 28 01:51:38 2018 From: dieter at handshake.de (dieter) Date: Wed, 28 Feb 2018 07:51:38 +0100 Subject: Questions about `locals` builtin References: <87inaivs8x.fsf@handshake.de> Message-ID: <87lgfd38ud.fsf@handshake.de> Kirill Balunov writes: > 2018-02-27 2:57 GMT+03:00 Terry Reedy : > >> The point of point 3 is that terminology and details would likely be >> different if Python were freshly designed more or less as it is today, and >> some things only make more or less sense in historical context. Learn what >> you need to know to write code that works. >> > > Thank you, I'm fairly familiar with the scope and namespace concepts in > Python 3, and they are also very well described in the "Execution model" > https://docs.python.org/3/reference/executionmodel.html#execution-model, > for this special thanks to the person who wrote it ;-) > > I started using Python with ?Python 3.5 and I'm not familiar with the > Python 2 features. But since Python 2 got into our discussion, I still have > a question: > > a. Is this restriction for locals desirable in the implementation of > CPython in Python 3? > b. Or is it the result of temporary fixes for Python 2? I think it is an implementation artefact: for efficiency reasons, local variables in a function are handled differently than "local" variables elsewhere. A side effect of the concrete implementation causes that truely local variables cannot be changed by updating the result of "locals()". The "locals" documentation tries to document the restrictions. From dieter at handshake.de Wed Feb 28 01:54:30 2018 From: dieter at handshake.de (dieter) Date: Wed, 28 Feb 2018 07:54:30 +0100 Subject: Questions about `locals` builtin References: <87inaivs8x.fsf@handshake.de> <1322f99f-609b-2eac-48a9-9d9be5d08efc@nedbatchelder.com> Message-ID: <87h8q138pl.fsf@handshake.de> Ned Batchelder writes: > On 2/27/18 3:52 AM, Kirill Balunov wrote: >> a. Is this restriction for locals desirable in the implementation of >> CPython in Python 3? >> b. Or is it the result of temporary fixes for Python 2? > > My understanding is that the behavior of locals() is determined mostly > by what is convenient for the implementors, so that they can keep > regular code running as quickly as possible.? The answer to the > question, "why can't we make locals() work more like I expect?" is, > "because that would make things slower." >> >> Personally, I find the convenient functionality to update the local symbol >> table inside a function, similar to `globals`. > > Can you show us an example of why you would want to update locals > through locals()?? There might be more natural ways to solve your > problem. I am still working with Python 2 (Python 3 may behave differently). There, during debugging, I would sometimes like to change the value of variables (I know that the variable has got a wrong value and would like to fix it to continue senseful debugging without a restart). This works for variables not yet known inside the function but does not work for true local variables. I assume that this is one effect of the "locals()" restriction. From dieter at handshake.de Wed Feb 28 01:57:48 2018 From: dieter at handshake.de (dieter) Date: Wed, 28 Feb 2018 07:57:48 +0100 Subject: In Python2, does it need to wrap imp.find/load_module with imp_acquire/release_lock? References: <71381ce7-213b-460e-b5d2-07337158d1ff@googlegroups.com> Message-ID: <87d10p38k3.fsf@handshake.de> Xiang Zhang writes: > Just like the title. It seems to me it is needed from the source code but codes in stdlib all doesn't do that. The "import" machinery uses locks of its own (to protect "sys.modules"). I assume that "load_module" will use those locks internally. "find" might be considered to usually see quite a static situation; should your file system be modified by concurrent threads, it might be necessary that you use your own locks to protect against this. From rosuav at gmail.com Wed Feb 28 02:01:42 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Feb 2018 18:01:42 +1100 Subject: Questions about `locals` builtin In-Reply-To: References: Message-ID: On Tue, Feb 27, 2018 at 5:55 AM, Kirill Balunov wrote: > 2. The documentation has a note that "The contents of this dictionary > should not be modified". Which implies that it is a read only mapping. So > the question why it is `dict` instead of `types.MappingProxyType`? A dict is smaller and faster. In some Python implementations, locals() returns an actual dictionary of the actual locals; to construct a proxy would be a waste of effort, and to have the internal locals use a proxy is also a complete waste. Python tends not to enforce rules that won't actually cause a crash, so it's simpler and more efficient to give you a real dictionary and say "changing this might not do what you think it does". > 3. There is one more moment: local variables had been determined when > function was compiled. But `locals` returns _some_ current runtime copy. I > find this also confusing: > > >>>> def func1(): > > loc = locals() > > b = 12 > > return loc > > >>>> def func2(): > > b = 12 > > loc = locals() > > return loc > > >>>> func1() > { } >>>> func2() > {'b': 12} The values of those locals is calculated at run-time, but the set of names is not. If a name has not been bound, it's still flagged as "local", but it doesn't appear in locals(), because there's no corresponding value. So if "b" is not in locals(), attempting to access the variable named b will raise UnboundLocalError. Dictionaries don't have a way to say "this thing exists but has no value"; so this says "this thing doesn't exist", which is close enough. If you really want a list of ALL the local names in a function, you can look at its __code__ object, which has a tuple of variable names: print(func1.__code__.co_varnames) That information is static to the function, as it is indeed determined when the function is compiled. ChrisA From rosuav at gmail.com Wed Feb 28 02:04:11 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 28 Feb 2018 18:04:11 +1100 Subject: Questions about `locals` builtin In-Reply-To: <87h8q138pl.fsf@handshake.de> References: <87inaivs8x.fsf@handshake.de> <1322f99f-609b-2eac-48a9-9d9be5d08efc@nedbatchelder.com> <87h8q138pl.fsf@handshake.de> Message-ID: On Wed, Feb 28, 2018 at 5:54 PM, dieter wrote: > Ned Batchelder writes: >> On 2/27/18 3:52 AM, Kirill Balunov wrote: >>> a. Is this restriction for locals desirable in the implementation of >>> CPython in Python 3? >>> b. Or is it the result of temporary fixes for Python 2? >> >> My understanding is that the behavior of locals() is determined mostly >> by what is convenient for the implementors, so that they can keep >> regular code running as quickly as possible. The answer to the >> question, "why can't we make locals() work more like I expect?" is, >> "because that would make things slower." >>> >>> Personally, I find the convenient functionality to update the local symbol >>> table inside a function, similar to `globals`. >> >> Can you show us an example of why you would want to update locals >> through locals()? There might be more natural ways to solve your >> problem. > > I am still working with Python 2 (Python 3 may behave differently). > There, during debugging, I would sometimes like to change the value > of variables (I know that the variable has got a wrong value > and would like to fix it to continue senseful debugging without a restart). > This works for variables not yet known inside the function but does > not work for true local variables. > I assume that this is one effect of the "locals()" restriction. > That seems like a hairy thing to do, honestly. But if you know that there's only a handful of variables that you'd actually want to do that to, you can simply put those into an object of some form, and then mutate that object. That's guaranteed to work in any Python. Personally, I'd be inclined to break things up into separate functions, and then if you want to change state and continue, it would be by interactively calling one of those functions with slightly different parameters. ChrisA From robin at reportlab.com Wed Feb 28 04:53:09 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 28 Feb 2018 09:53:09 +0000 Subject: py2exe output flagged as malware Message-ID: <75327cd1-9d58-7610-4e70-7af6977c3af0@chamonix.reportlab.co.uk> I see this has happened to others in the past. I'm using 32 bit python 2.7.10 with py2exe 3.3 on windows 7. The exes work fine, but when I try to download into windows 10 I'm getting the exes immediately removed as malware. Is there any way to prevent this. It's very bad for python programs to get this kind of reputation. Anyone know if the same happens with PyInstaller etc etc? -- Robin Becker From alister.ware at ntlworld.com Wed Feb 28 06:46:19 2018 From: alister.ware at ntlworld.com (alister) Date: Wed, 28 Feb 2018 11:46:19 GMT Subject: py2exe output flagged as malware References: <75327cd1-9d58-7610-4e70-7af6977c3af0@chamonix.reportlab.co.uk> Message-ID: On Wed, 28 Feb 2018 09:53:09 +0000, Robin Becker wrote: > I see this has happened to others in the past. I'm using 32 bit python > 2.7.10 with py2exe 3.3 on windows 7. The exes work fine, > but when I try to download into windows 10 I'm getting the exes > immediately removed as malware. > > Is there any way to prevent this. It's very bad for python programs to > get this kind of reputation. > > Anyone know if the same happens with PyInstaller etc etc? I would suggest you contact the Anti-virus vendor, Are you sure you have download a clean copy of py2exe. -- There are two ways to write error-free programs; only the third one works. From steve+comp.lang.python at pearwood.info Wed Feb 28 06:58:45 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 28 Feb 2018 11:58:45 +0000 (UTC) Subject: Questions about `locals` builtin References: <87inaivs8x.fsf@handshake.de> <1322f99f-609b-2eac-48a9-9d9be5d08efc@nedbatchelder.com> <87h8q138pl.fsf@handshake.de> Message-ID: On Wed, 28 Feb 2018 18:04:11 +1100, Chris Angelico wrote: > On Wed, Feb 28, 2018 at 5:54 PM, dieter wrote: [...] >> I am still working with Python 2 (Python 3 may behave differently). >> There, during debugging, I would sometimes like to change the value of >> variables (I know that the variable has got a wrong value and would >> like to fix it to continue senseful debugging without a restart). This >> works for variables not yet known inside the function but does not work >> for true local variables. >> I assume that this is one effect of the "locals()" restriction. >> >> > That seems like a hairy thing to do, honestly. Well, yeah, but it is during debugging, not production code. > But if you know that > there's only a handful of variables that you'd actually want to do that > to, you can simply put those into an object of some form, and then > mutate that object. o_O If I knew ahead of time which variables held the wrong value, I'd fix them so they held the right value ahead of time, and not bother using a debugger. :-) -- Steve From steve+comp.lang.python at pearwood.info Wed Feb 28 06:59:56 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 28 Feb 2018 11:59:56 +0000 (UTC) Subject: Questions about `locals` builtin References: Message-ID: On Wed, 28 Feb 2018 18:01:42 +1100, Chris Angelico wrote: > If you really want a list of ALL the local names in a function, you can > look at its __code__ object, which has a tuple of variable names: > > print(func1.__code__.co_varnames) > > That information is static to the function, as it is indeed determined > when the function is compiled. Ho ho ho, not in Python 2 it isn't!!! py> def bizarre(): ... x = 1 ... from math import * # oww my aching head! ... print 'sin' in locals() ... print sin ... :1: SyntaxWarning: import * only allowed at module level py> bizarre() True py> bizarre.__code__.co_varnames ('x',) In Python 2, you can also use exec inside a function, for extra obfuscatory goodness. Python 3 locks down these loopholes, and ensures that locals inside functions can be statically determined. -- Steve From rosuav at gmail.com Wed Feb 28 08:51:09 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 00:51:09 +1100 Subject: Questions about `locals` builtin In-Reply-To: References: <87inaivs8x.fsf@handshake.de> <1322f99f-609b-2eac-48a9-9d9be5d08efc@nedbatchelder.com> <87h8q138pl.fsf@handshake.de> Message-ID: On Wed, Feb 28, 2018 at 10:58 PM, Steven D'Aprano wrote: > On Wed, 28 Feb 2018 18:04:11 +1100, Chris Angelico wrote: >> But if you know that >> there's only a handful of variables that you'd actually want to do that >> to, you can simply put those into an object of some form, and then >> mutate that object. > > o_O > > If I knew ahead of time which variables held the wrong value, I'd fix > them so they held the right value ahead of time, and not bother using a > debugger. > > :-) Ahh but that requires that you know the right value as well. It's plausible that you have a handful of "tweakables" but you have no idea what to tweak them *to*. That's where this would be plausible. ChrisA From rosuav at gmail.com Wed Feb 28 08:54:23 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 00:54:23 +1100 Subject: Questions about `locals` builtin In-Reply-To: References: Message-ID: On Wed, Feb 28, 2018 at 10:59 PM, Steven D'Aprano wrote: > On Wed, 28 Feb 2018 18:01:42 +1100, Chris Angelico wrote: > >> If you really want a list of ALL the local names in a function, you can >> look at its __code__ object, which has a tuple of variable names: >> >> print(func1.__code__.co_varnames) >> >> That information is static to the function, as it is indeed determined >> when the function is compiled. > > Ho ho ho, not in Python 2 it isn't!!! > > py> def bizarre(): > ... x = 1 > ... from math import * # oww my aching head! > ... print 'sin' in locals() > ... print sin > ... The sin in question is that you're using a star import inside a function. The penance is to kneel before the altar and recite PEP 20 nineteen times. And then cut down the tallest tree in the forest... with... a debugger. (When I tried that, I got a SyntaxWarning. Yay!) ChrisA From larry.martell at gmail.com Wed Feb 28 08:55:19 2018 From: larry.martell at gmail.com (Larry Martell) Date: Wed, 28 Feb 2018 08:55:19 -0500 Subject: psutil In-Reply-To: References: <20180228003627.GB6492@miniequipaje> Message-ID: On Tue, Feb 27, 2018 at 8:30 PM, Matt Wheeler wrote: > > > On Wed, 28 Feb 2018, 00:49 Larry Martell, wrote: >> >> On Tue, Feb 27, 2018 at 7:36 PM, Jos? Mar?a Mateos >> wrote: >> > On Tue, Feb 27, 2018 at 07:29:50PM -0500, Larry Martell wrote: >> >> Trying to install psutil (with pip install psutil) on Red Hat EL 7. >> >> It's failing with: >> >> >> >> Python.h: No such file or directory >> > >> > Two questions come to my mind: >> > >> > - Does it work if you try to install some other package? >> >> Yes, I have installed other packages (e.g. numpy) > > > Did that install definitely involve a build from source? > >> > - Is `pip` by any change trying to install a Python 3 package, but you >> > only have the libraries for Python 2 installed? >> >> No, here is the gcc command line that failed: >> >> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall >> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong >> --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic >> -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall >> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong >> --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic >> -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 >> -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 >> -I/opt/rh/python27/root/usr/include/python2.7 -c >> psutil/_psutil_common.c -o >> build/temp.linux-x86_64-2.7/psutil/_psutil_common.o > > > Based on the include path you have here this looks like you have a non > standard python package for EL7 (in addition to the system one, or you'd be > having a worse day), which is probably the root of your problem. > > Are you using python packages from scl perhaps? > If so you probably need to install python27-python-devel (found at > http://mirror.centos.org/centos/7/sclo/x86_64/rh/python27/) Thanks to all who replied. The machine got rebooted (for an unrelated issue) and when it came back up the install worked fine. From robin at reportlab.com Wed Feb 28 10:47:50 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 28 Feb 2018 15:47:50 +0000 Subject: py2exe output flagged as malware In-Reply-To: References: <75327cd1-9d58-7610-4e70-7af6977c3af0@chamonix.reportlab.co.uk> Message-ID: <9ec77558-42ea-cbd1-eb40-f47e26f0fa11@chamonix.reportlab.co.uk> On 28/02/2018 11:46, alister via Python-list wrote: > On Wed, 28 Feb 2018 09:53:09 +0000, Robin Becker wrote: > >> I see this has happened to others in the past. I'm using 32 bit python >> 2.7.10 with py2exe 3.3 on windows 7. The exes work fine, >> but when I try to download into windows 10 I'm getting the exes >> immediately removed as malware. >> >> Is there any way to prevent this. It's very bad for python programs to >> get this kind of reputation. >> >> Anyone know if the same happens with PyInstaller etc etc? > > I would suggest you contact the Anti-virus vendor, > Are you sure you have download a clean copy of py2exe. The anti-virus vendor is Microsoft; I imagine I will get short shrift from them regarding this issue. Turns out my py2exe script was just pyinstaller under the hood. Apologies to py2exe. I guess someone changed the build script script to use a different python packager. I upgraded pyinstaller using the very latest pip and now the version of pyinstaller at least is 3.3.1. I don't actually know how to check the validity of the installed code or the binary stubs. After rebuilding with 3.3.1 the new exe is 10 bytes shorter and is not recognized as malware by windows 10. -- Robin Becker From h.goebel at goebel-consult.de Wed Feb 28 11:24:39 2018 From: h.goebel at goebel-consult.de (Hartmut Goebel) Date: Wed, 28 Feb 2018 17:24:39 +0100 Subject: py2exe output flagged as malware In-Reply-To: <9ec77558-42ea-cbd1-eb40-f47e26f0fa11@chamonix.reportlab.co.uk> References: <75327cd1-9d58-7610-4e70-7af6977c3af0@chamonix.reportlab.co.uk> <9ec77558-42ea-cbd1-eb40-f47e26f0fa11@chamonix.reportlab.co.uk> Message-ID: <0f65e3fc-012b-107d-3a09-0cc58be9f403@goebel-consult.de> Am 28.02.2018 um 16:47 schrieb Robin Becker: > I upgraded pyinstaller using the very latest pip and now the version > of pyinstaller at least is 3.3.1. I don't actually know how to check > the validity of the installed code or the binary stubs. The current archives are PyPI are PGP/GnuPG-signed, as the ones at github. -- Sch?nen Gru? Hartmut Goebel Dipl.-Informatiker (univ), CISSP, CSSLP, ISO 27001 Lead Implementer Information Security Management, Security Governance, Secure Software Development Goebel Consult, Landshut http://www.goebel-consult.de Blog: http://www.goebel-consult.de/blog/nicht-pgp-ist-gescheitert-die-entwickler-haben-versagt Kolumne: http://www.cissp-gefluester.de/2012-02-bring-your-own-life-glosse From h.goebel at crazy-compilers.com Wed Feb 28 11:25:15 2018 From: h.goebel at crazy-compilers.com (Hartmut Goebel) Date: Wed, 28 Feb 2018 17:25:15 +0100 Subject: py2exe output flagged as malware In-Reply-To: <9ec77558-42ea-cbd1-eb40-f47e26f0fa11@chamonix.reportlab.co.uk> References: <75327cd1-9d58-7610-4e70-7af6977c3af0@chamonix.reportlab.co.uk> <9ec77558-42ea-cbd1-eb40-f47e26f0fa11@chamonix.reportlab.co.uk> Message-ID: <729cd4ef-b959-203f-459c-fc59c473edbf@crazy-compilers.com> Am 28.02.2018 um 16:47 schrieb Robin Becker: > I upgraded pyinstaller using the very latest pip and now the version > of pyinstaller at least is 3.3.1. I don't actually know how to check > the validity of the installed code or the binary stubs. The current archives are PyPI are PGP/GnuPG-signed, as the ones at github. -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel at crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | From robin at reportlab.com Wed Feb 28 12:07:49 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 28 Feb 2018 17:07:49 +0000 Subject: py2exe output flagged as malware In-Reply-To: <729cd4ef-b959-203f-459c-fc59c473edbf@crazy-compilers.com> References: <75327cd1-9d58-7610-4e70-7af6977c3af0@chamonix.reportlab.co.uk> <9ec77558-42ea-cbd1-eb40-f47e26f0fa11@chamonix.reportlab.co.uk> <729cd4ef-b959-203f-459c-fc59c473edbf@crazy-compilers.com> Message-ID: <8fd4d408-ad35-3f68-9aac-aee45a792d06@chamonix.reportlab.co.uk> On 28/02/2018 16:25, Hartmut Goebel wrote: > Am 28.02.2018 um 16:47 schrieb Robin Becker: >> I upgraded pyinstaller using the very latest pip and now the version >> of pyinstaller at least is 3.3.1. I don't actually know how to check >> the validity of the installed code or the binary stubs. > > The current archives are PyPI are PGP/GnuPG-signed, as the ones at github. > so presumably I can locate the downloaded tar zip and hash it to check. Of course if pip is already busted it can out fox me anyhow. -- Robin Becker From jrlcgmx at gmail.com Wed Feb 28 12:50:28 2018 From: jrlcgmx at gmail.com (jrlcgmx at gmail.com) Date: Wed, 28 Feb 2018 09:50:28 -0800 (PST) Subject: Which sites allow you to learn interactively Python3 Message-ID: <7b02c161-ec7e-408f-9322-26bfc7056f1e@googlegroups.com> Which sites allow you to learn interactively Python3? they can be payed, actually payed would probably be bettter Good quality source From jrlcgmx at gmail.com Wed Feb 28 12:58:24 2018 From: jrlcgmx at gmail.com (Aktive) Date: Wed, 28 Feb 2018 09:58:24 -0800 (PST) Subject: help me ? (Posting On Python-List Prohibited) In-Reply-To: <540c622c-e40d-4f29-a6cd-577c535b248e@googlegroups.com> References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> <04591b86-19e7-46c8-9ffc-0b3980769213@googlegroups.com> <540c622c-e40d-4f29-a6cd-577c535b248e@googlegroups.com> Message-ID: <158fa187-51f4-4368-926e-0a002440c0cf@googlegroups.com> On Wednesday, 28 February 2018 06:00:07 UTC+1, Lawrence D?Oliveiro wrote: > On Wednesday, February 28, 2018 at 4:44:04 PM UTC+13, jlad... at itu.edu wrote: > > This is part of the reason why interviews for software developer jobs > > have gotten so crazy. > > This is why you need to have a CV that basically says ?Google me?. what the hell do you care about cheating.. the world doest care about cheating. its about skill. You guys been too much in school From arj.python at gmail.com Wed Feb 28 12:59:48 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Wed, 28 Feb 2018 21:59:48 +0400 Subject: Which sites allow you to learn interactively Python3 In-Reply-To: <7b02c161-ec7e-408f-9322-26bfc7056f1e@googlegroups.com> References: <7b02c161-ec7e-408f-9322-26bfc7056f1e@googlegroups.com> Message-ID: see clever programmer python by projects. Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ On 28 Feb 2018 21:56, wrote: > Which sites allow you to learn interactively Python3? > > they can be payed, > > actually payed would probably be bettter > Good quality source > -- > https://mail.python.org/mailman/listinfo/python-list > From fabiofz at gmail.com Wed Feb 28 13:57:17 2018 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Wed, 28 Feb 2018 15:57:17 -0300 Subject: PyDev 6.3.1 Released Message-ID: PyDev 6.3.1 Release Highlights - PyDev is now also available for Python coding on Visual Studio Code -- see: http://www.pydev.org/vscode/ for more details. PyDev changes: - Type inference - Folders no longer require *__init__* to be considered a package. - Properly recognize *cx_Oracle.cp36-win_amd64.pyd* as *cx_Oracle* ( *#PyDev-885*). - Empty numpy arrays properly handled in debugger. - Fix to get path to activate conda env on Linux. - Fix debug console freeze when evaluation raises exception with Python 3.5 onwards (*#PyDev-877*). - Interactive console accepting new args passed by IPython in showtraceback (*#PyDev-882*). - Improve terminating running processes (and children). - Properly parsing f-strings which contain double *{{* or *}}* ( *#PyDev-884*). About PyDev PyDev is an open-source Python IDE on top of Eclipse for Python, Jython and IronPython development, now also available for Python on Visual Studio Code. It comes with goodies such as code completion, syntax highlighting, syntax analysis, code analysis, refactor, debug, interactive console, etc. It is also available as a standalone through LiClipse with goodies such as Multiple cursors, theming and support for many other languages, such as Django Templates, Jinja2, Html, JavaScript, etc. Links: PyDev: http://pydev.org PyDev Blog: http://pydev.blogspot.com PyDev on VSCode: http://pydev.org/vscode LiClipse: http://www.liclipse.com PyVmMonitor - Python Profiler: http://www.pyvmmonitor.com/ Cheers, Fabio Zadrozny From jrlcgmx at gmail.com Wed Feb 28 14:55:49 2018 From: jrlcgmx at gmail.com (jrlcgmx at gmail.com) Date: Wed, 28 Feb 2018 11:55:49 -0800 (PST) Subject: cute interview problem In-Reply-To: <87a7vv71zg.fsf@nightsong.com> References: <87a7vv71zg.fsf@nightsong.com> Message-ID: <7c9db8d0-64d3-4793-b9a1-c3274f602f85@googlegroups.com> On Tuesday, 27 February 2018 00:42:02 UTC+1, Paul Rubin wrote: > Ron Aaron posted the below url on comp.lang.forth. It points to what I > thought was a cute problem, along with his solution in his Forth dialect > 8th: > > https://8th-dev.com/forum/index.php/topic,1584.msg8720.html > > I wrote a solution in Forth and additional ones in Python and Haskell > and thought people here might like trying it themselves. I'll post my > Python version here in a few days if anyone wants to see it. Time limit > for the problem is supposed to be 45 minutes. I spent a lot longer > because I ended up writing several versions in various languages. I dont think its cute at all. how did the interview go? From jrlcgmx at gmail.com Wed Feb 28 14:56:28 2018 From: jrlcgmx at gmail.com (jrlcgmx at gmail.com) Date: Wed, 28 Feb 2018 11:56:28 -0800 (PST) Subject: cute interview problem In-Reply-To: <87a7vv71zg.fsf@nightsong.com> References: <87a7vv71zg.fsf@nightsong.com> Message-ID: On Tuesday, 27 February 2018 00:42:02 UTC+1, Paul Rubin wrote: > Ron Aaron posted the below url on comp.lang.forth. It points to what I > thought was a cute problem, along with his solution in his Forth dialect > 8th: > > https://8th-dev.com/forum/index.php/topic,1584.msg8720.html > > I wrote a solution in Forth and additional ones in Python and Haskell > and thought people here might like trying it themselves. I'll post my > Python version here in a few days if anyone wants to see it. Time limit > for the problem is supposed to be 45 minutes. I spent a lot longer > because I ended up writing several versions in various languages. teh interview how did it go From ian.g.kelly at gmail.com Wed Feb 28 15:51:02 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 28 Feb 2018 13:51:02 -0700 Subject: cute interview problem In-Reply-To: <7c9db8d0-64d3-4793-b9a1-c3274f602f85@googlegroups.com> References: <87a7vv71zg.fsf@nightsong.com> <7c9db8d0-64d3-4793-b9a1-c3274f602f85@googlegroups.com> Message-ID: On Wed, Feb 28, 2018 at 12:55 PM, wrote: > On Tuesday, 27 February 2018 00:42:02 UTC+1, Paul Rubin wrote: >> Ron Aaron posted the below url on comp.lang.forth. It points to what I >> thought was a cute problem, along with his solution in his Forth dialect >> 8th: >> >> https://8th-dev.com/forum/index.php/topic,1584.msg8720.html >> >> I wrote a solution in Forth and additional ones in Python and Haskell >> and thought people here might like trying it themselves. I'll post my >> Python version here in a few days if anyone wants to see it. Time limit >> for the problem is supposed to be 45 minutes. I spent a lot longer >> because I ended up writing several versions in various languages. > > I dont think its cute at all. > how did the interview go? Yeah, seems to me this is basically just asking "have you ever heard of an interval tree (or can you invent one on the fly)". From tkadm30 at yandex.com Wed Feb 28 16:13:52 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 28 Feb 2018 16:13:52 -0500 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> Message-ID: Hey, I want to know why this question is being silently ignored by this group. I think it is essential to develop cryptoanalysis tools (like softwares) to detect and block ultrasonic signals in mobile devices. Using Python to detect or block ultrasonic side channels would be very interesting. I want to know what exactly do I need to develop open source softwares for analyzing and decoding ultrasonic side channels in smart tvs, mobile phones, and computers in Python. Please speak your mind out... Tell me how exactly ultrasonic side channels may activate remotely specific neural pathways implicated in aggressivity and how to block theses specific side channels from neuromodulating human behavior. Thank you. Etienne Le 2018-02-26 ? 05:25, Etienne Robillard a ?crit?: > Hi, > > I would like to know if its possible to detect or decode ultrasonic > signals in mobiles devices like Android with Python? > > For a introduction to ultrasonic tracking, see: > https://www.wired.com/2016/11/block-ultrasonic-signals-didnt-know-tracking/ > > Thank you, > > Etienne > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From ned at nedbatchelder.com Wed Feb 28 16:49:27 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 28 Feb 2018 16:49:27 -0500 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> Message-ID: <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> On 2/28/18 4:13 PM, Etienne Robillard wrote: > I want to know why this question is being silently ignored by this group. If no one has any information about your topic, then no one will say anything.? Python on Android is very specialized as it is, and I have no idea what ultrasonic side channels are, much less what it would take to detect or block them. There's no guarantee that any given question will get a response, much less a satisfactory answer.? The silence is not malicious, you just haven't piqued anyone's interest enough for them to respond. --Ned. From rosuav at gmail.com Wed Feb 28 16:54:52 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 08:54:52 +1100 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> Message-ID: On Thu, Mar 1, 2018 at 8:49 AM, Ned Batchelder wrote: > On 2/28/18 4:13 PM, Etienne Robillard wrote: >> >> I want to know why this question is being silently ignored by this group. > > > If no one has any information about your topic, then no one will say > anything. Python on Android is very specialized as it is, and I have no > idea what ultrasonic side channels are, much less what it would take to > detect or block them. > > There's no guarantee that any given question will get a response, much less > a satisfactory answer. The silence is not malicious, you just haven't > piqued anyone's interest enough for them to respond. > I rather suspect that this is more an Android question than a Python one, so there's likely to be more interest in an Android community. But this part sounds like prime quality tinfoil hat material: > Tell me how exactly ultrasonic side channels may activate remotely specific > neural pathways implicated in aggressivity and how to block theses specific > side channels from neuromodulating human behavior. Should be easy to find some whack-job newsgroups that would love to discuss that aspect of it. ChrisA From gordon at panix.com Wed Feb 28 17:06:30 2018 From: gordon at panix.com (John Gordon) Date: Wed, 28 Feb 2018 22:06:30 +0000 (UTC) Subject: Detection of ultrasonic side channels in mobile devices with Python? References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> Message-ID: In Chris Angelico writes: > > Tell me how exactly ultrasonic side channels may activate remotely specific > > neural pathways implicated in aggressivity and how to block theses specific > > side channels from neuromodulating human behavior. > Should be easy to find some whack-job newsgroups that would love to > discuss that aspect of it. Sounds like the plot to the latest Kingsman movie. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From tkadm30 at yandex.com Wed Feb 28 17:31:34 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 28 Feb 2018 17:31:34 -0500 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> Message-ID: If ultrasonic side channels are a threat to privacy, it is because smart phones were designed to exploit this precise feature. A great number of studies have shown that ultrasonic neuromodulation of the central nervous system can be exploited via brain-computer interfaces... It is cutting edge science however, and my knowledge on techniques for activating human behavior by remote means is limited. I am only an amateur scientist/programmer with no experience whatsoever in Android or neuroscience. However, the possibility of ultrasonic side channels to be exploited by a third-party remotely without the user consent seem highly likely. I want to develop Python tools to provide experimental evidences that Android may exploit ultrasonic side channels to stimulate or impair the auditory cortex of someone. In other words, I plan to disclose the ability of the US government to use mobile devices as psychoenergetics (non-lethal) bioweapons. Etienne Le 2018-02-28 ? 16:54, Chris Angelico a ?crit?: > On Thu, Mar 1, 2018 at 8:49 AM, Ned Batchelder wrote: >> On 2/28/18 4:13 PM, Etienne Robillard wrote: >>> I want to know why this question is being silently ignored by this group. >> >> If no one has any information about your topic, then no one will say >> anything. Python on Android is very specialized as it is, and I have no >> idea what ultrasonic side channels are, much less what it would take to >> detect or block them. >> >> There's no guarantee that any given question will get a response, much less >> a satisfactory answer. The silence is not malicious, you just haven't >> piqued anyone's interest enough for them to respond. >> > I rather suspect that this is more an Android question than a Python > one, so there's likely to be more interest in an Android community. > > But this part sounds like prime quality tinfoil hat material: > >> Tell me how exactly ultrasonic side channels may activate remotely specific >> neural pathways implicated in aggressivity and how to block theses specific >> side channels from neuromodulating human behavior. > Should be easy to find some whack-job newsgroups that would love to > discuss that aspect of it. > > ChrisA -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From ooomzay at gmail.com Wed Feb 28 17:51:09 2018 From: ooomzay at gmail.com (ooomzay at gmail.com) Date: Wed, 28 Feb 2018 14:51:09 -0800 (PST) Subject: RFC: Proposal: Deterministic Object Destruction Message-ID: Informal Background =================== Python's lack of Deterministic Object Destruction semantics strikes me as very unpythonic. This state of affairs spawns lengthy diatribes on __del__(), a variety of specialised and onerous resource management solutions and malignant carbuncles such as PEP 343 (the "with" statement). None of these match the practical utility, simplicity, generality and robustness of Deterministic Object Destruction combined with the RAII idiom to manage resources. I have therefore drafted the radical but simple and 100% backward compatible proposal below to remedy this situation and pave the way for numerous tidy ups (out of scope of this proposal). I am now retiring to a safe distance and invite your comments on the PEP below:- Abstract ======== This PEP proposes that valid python interpreters *must* synchronously destroy objects when the last reference to an object goes out of scope. This interpreter behaviour is currently permitted and exhibited by the reference implementation [CPython], but it is optional. Motivation ========== To make the elegance (simplicity, generality, and robustness) of the "Resource Acquisition Is Initialization" (RAII) idiom available to all python applications and libraries. To render less elegant and more onerous resource management solutions, such as the "with" Statement [PEP 343] obsolete. Specification ============= When the last reference to an object goes out of scope the intepreter must synchronously, in the thread that releases the last reference, invoke the object's __del__() method and then free the memory occupied by that object. Backwards Compatibility ======================= This proposal preserves 100% backwards compatibility. The language currently does not prescribe when, or even if, __del()__ will be called and object memory freed. Existing python applications and libraries can make no assumptions about this behaviour. Reference Implementation ======================== CPython References ========== PEP 343 -- The "with" Statement From rosuav at gmail.com Wed Feb 28 18:01:42 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 10:01:42 +1100 Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: References: Message-ID: On Thu, Mar 1, 2018 at 9:51 AM, wrote: > Specification > ============= > > When the last reference to an object goes out of scope the intepreter must synchronously, in the thread that releases the last reference, invoke the object's __del__() method and then free the memory occupied by that object. > If it were that simple, why do you think it isn't currently mandated? Here's one example: reference cycles. When do they get detected? Taking a really simple situation: class Foo: def __init__(self): self.self = self print("Creating a Foo") def __del__(self): print("Disposing of a Foo") foo = Foo() foo = 1 When do you expect __del__ to be called? How do you implement this efficiently and reliably? ChrisA From steve+comp.lang.python at pearwood.info Wed Feb 28 18:23:57 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 28 Feb 2018 23:23:57 +0000 (UTC) Subject: Detection of ultrasonic side channels in mobile devices with Python? References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> Message-ID: On Wed, 28 Feb 2018 16:13:52 -0500, Etienne Robillard wrote: > Hey, > > I want to know why this question is being silently ignored by this > group. Possibly because we have no idea of the answer? Or we thought it was too obvious to answer? Do you have a microphone that can detect ultrasound? Can Python communicate with that microphone? If the answer to both of those questions is Yes, then the answer to your question is also yes. The answer to EVERY question "Can Python do Foo?" usually comes down to: - can you do Foo using a computer? then Python can do it too although it might not always be fast enough. [...] > I want to know what exactly do I need to develop open source softwares > for analyzing and decoding ultrasonic side channels in smart tvs, mobile > phones, and computers in Python. What makes you think we're experts on analyzing and decoding ultrasonic side channels in smart TVs, mobile phones, or computers? Let alone all three. > Tell me how exactly ultrasonic side channels may activate remotely > specific neural pathways implicated in aggressivity and how to block > theses specific side channels from neuromodulating human behavior. And what on Earth gave you the idea that we're experts on ultrasonic activation of neural pathways???? I'd go further... what gave you the idea that ANYONE is an expert on ultrasonic activation of neural pathways? -- Steve From rosuav at gmail.com Wed Feb 28 18:29:29 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 10:29:29 +1100 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> Message-ID: On Thu, Mar 1, 2018 at 10:23 AM, Steven D'Aprano wrote: > I'd go further... what gave you the idea that ANYONE is an expert on > ultrasonic activation of neural pathways? Definitely tinfoil hat material here; expertise is completely immaterial. ChrisA From johnpote at jptechnical.co.uk Wed Feb 28 18:40:41 2018 From: johnpote at jptechnical.co.uk (John Pote) Date: Wed, 28 Feb 2018 23:40:41 +0000 Subject: csv module and NULL data byte Message-ID: I have a csv data file that may become corrupted (already happened) resulting in a NULL byte appearing in the file. The NULL byte causes an _csv.Error exception. I'd rather like the csv reader to return csv lines as best it can and subsequent processing of each comma separated field deal with illegal bytes. That way as many lines from the file may be processed and the corrupted ones simply dumped. Is there a way of getting the csv reader to accept all 256 possible bytes. (with \r,\n and ',' bytes delimiting lines and fields). My test code is, ??? with open( fname, 'rt', encoding='iso-8859-1' ) as csvfile: ??? ??? csvreader = csv.reader(csvfile, delimiter=',', quoting=csv.QUOTE_NONE, strict=False ) ??? ??? ??? data = list( csvreader ) ??? ?? ?? ? for ln in data: ??? ??? ??? ??? print( ln ) Result >>python36 csvTest.py Traceback (most recent call last): ? File "csvTest.py", line 22, in ??? data = list( csvreader ) _csv.Error: line contains NULL byte strict=False or True makes no difference. Help appreciated, John From grant.b.edwards at gmail.com Wed Feb 28 18:43:22 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 28 Feb 2018 23:43:22 +0000 (UTC) Subject: Detection of ultrasonic side channels in mobile devices with Python? References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> Message-ID: On 2018-02-28, Chris Angelico wrote: > On Thu, Mar 1, 2018 at 10:23 AM, Steven D'Aprano > wrote: >> I'd go further... what gave you the idea that ANYONE is an expert on >> ultrasonic activation of neural pathways? > > Definitely tinfoil hat material here; expertise is completely immaterial. I miss Ludwig Plutonium. :/ -- Grant Edwards grant.b.edwards Yow! Nipples, dimples, at knuckles, NICKLES, gmail.com wrinkles, pimples!! From ooomzay at gmail.com Wed Feb 28 18:44:45 2018 From: ooomzay at gmail.com (ooomzay at gmail.com) Date: Wed, 28 Feb 2018 15:44:45 -0800 (PST) Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: References: Message-ID: <6ad254cb-7318-4e24-8515-cedf9ab9feb6@googlegroups.com> On Wednesday, February 28, 2018 at 11:02:17 PM UTC, Chris Angelico wrote: > On Thu, Mar 1, 2018 at 9:51 AM, ooomzay wrote: > > Specification > > ============= > > > > When the last reference to an object goes out of scope the intepreter must synchronously, in the thread that releases the last reference, invoke the object's __del__() method and then free the memory occupied by that object. > > > > If it were that simple, why do you think it isn't currently mandated? We know the implementation is feasible because CPython, the reference interpreter implementation, already does this. I am guessing that there is a lobby from garbage platforms where it would be harder to implement and integrate however the object of this proposal is the integrity of the language, not on making it easier to implement using garbage platforms. > Here's one example: reference cycles. When do they get detected? Orphan cycle _detection_ is orthogonal to this proposal. As cycles are always a symptom of a design error I personally have little interest in them other than some facility to detect/debug and eliminate them from applications by design (another topic). > Taking a really simple situation: > > class Foo: > def __init__(self): > self.self = self > print("Creating a Foo") > def __del__(self): > print("Disposing of a Foo") > > foo = Foo() > foo = 1 > > When do you expect __del__ to be called? At the point of (re)assignment: "foo = 1" > How do you implement this efficiently and reliably? CPython already has a reliable and reasonably efficient implementation based on reference counting. From steve+comp.lang.python at pearwood.info Wed Feb 28 18:48:23 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 28 Feb 2018 23:48:23 +0000 (UTC) Subject: RFC: Proposal: Deterministic Object Destruction References: Message-ID: On Wed, 28 Feb 2018 14:51:09 -0800, ooomzay wrote: > Informal Background > =================== > > Python's lack of Deterministic Object Destruction semantics strikes me > as very unpythonic. This state of affairs spawns lengthy diatribes on > __del__(), I've never seen any of these lengthy diatribes. If they're on your blog, you may be over-estimating how many readers you have. > malignant carbuncles such as PEP 343 (the "with"statement). Ah, this is going to be one of *those* posts. > This PEP proposes that valid python interpreters *must* synchronously > destroy objects when the last reference to an object goes out of scope. Well, that rules out any practical Python interpreter running on the Java or .Net virtual machines, not to mention banning advanced and modern garbage collectors unless they also include a reference counter. So that's Jython and IronPython out, and PyPy's useful work on alternative garbage collectors out the window. It also enshrines the GIL (or some other locking mechanism) as not just an implementation detail of CPython, but as mandatory for all Python interpreters. > Motivation > ========== > > To make the elegance (simplicity, generality, and robustness) of the > "Resource Acquisition Is Initialization" (RAII) idiom available to all > python applications and libraries. I've never yet seen an explanation of RAII that makes the least amount of sense. I wouldn't call it either elegant or simple. More like "confusing and badly named". > Specification > ============= > > When the last reference to an object goes out of scope the intepreter > must synchronously, in the thread that releases the last reference, > invoke the object's __del__() method and then free the memory occupied > by that object. What happens if the __del__ method recreates a reference to the object? > Backwards Compatibility > ======================= > > This proposal preserves 100% backwards compatibility. So long as you're not using IronPython, Jython, or some alternate builds of PyPy. -- Steve From steve+comp.lang.python at pearwood.info Wed Feb 28 18:51:43 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 28 Feb 2018 23:51:43 +0000 (UTC) Subject: Detection of ultrasonic side channels in mobile devices with Python? References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> Message-ID: On Thu, 01 Mar 2018 08:54:52 +1100, Chris Angelico wrote: > But this part sounds like prime quality tinfoil hat material: > >> Tell me how exactly ultrasonic side channels may activate remotely >> specific neural pathways implicated in aggressivity and how to block >> theses specific side channels from neuromodulating human behavior. I wouldn't entirely rule it out. (On the other hand, I wouldn't give it much credence in the absence of solid evidence either.) I'm too lazy to google it right now, but I recall that there's definitely evidence for inaudible *subsonic* frequencies being able to affect human emotions, mostly to induce feelings of disquiet and nameless dread. I recall a panel of US scientists in the 1980s trying to work out how to protect long-lived nuclear waste from inquisitive humans for periods of many millions of years. Obviously Keep Out signs aren't going to do the job, so one of the ideas they were tossing around was to surround the area in artificial rock carved into shapes which would generate the appropriate subsonic frequencies when the wind blew. Of course it is only a matter of time before malware, spyware, adware and hostile-national-security-ware[1] takes advantage of ultrasonic side channels to screw us over however they can. Including the US government. But I think the idea that the mobile phone manufacturers *intentionally* built phones with this capability in order to give the US government the ability to weaponise people's phones against them is sheer nuttiness. [1] In the current political climate, I think it is safe to say that nearly all national security ware is hostile, and every one of us are the targets. -- Steve From steve+comp.lang.python at pearwood.info Wed Feb 28 18:52:00 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 28 Feb 2018 23:52:00 +0000 (UTC) Subject: help me ? (Posting On Python-List Prohibited) References: <2edd557f-91ce-4b14-b79e-5a524596fd40@googlegroups.com> <04591b86-19e7-46c8-9ffc-0b3980769213@googlegroups.com> <540c622c-e40d-4f29-a6cd-577c535b248e@googlegroups.com> <158fa187-51f4-4368-926e-0a002440c0cf@googlegroups.com> Message-ID: On Wed, 28 Feb 2018 09:58:24 -0800, Aktive wrote: > what the hell do you care about cheating.. > > the world doest care about cheating. > > its about skill. Because cheaters don't have skill. That's why they cheat. > You guys been too much in school Ah, spoken like a cheater. -- Steve From rosuav at gmail.com Wed Feb 28 18:52:57 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 10:52:57 +1100 Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: <6ad254cb-7318-4e24-8515-cedf9ab9feb6@googlegroups.com> References: <6ad254cb-7318-4e24-8515-cedf9ab9feb6@googlegroups.com> Message-ID: On Thu, Mar 1, 2018 at 10:44 AM, wrote: > On Wednesday, February 28, 2018 at 11:02:17 PM UTC, Chris Angelico wrote: >> On Thu, Mar 1, 2018 at 9:51 AM, ooomzay wrote: >> > Specification >> > ============= >> > >> > When the last reference to an object goes out of scope the intepreter must synchronously, in the thread that releases the last reference, invoke the object's __del__() method and then free the memory occupied by that object. >> > >> >> If it were that simple, why do you think it isn't currently mandated? > > We know the implementation is feasible because CPython, the reference interpreter implementation, already does this. I am guessing that there is a lobby from garbage platforms where it would be harder to implement and integrate however the object of this proposal is the integrity of the language, not on making it easier to implement using garbage platforms. > Good to know that Jython and friends are "garbage platforms". I hope you were referring to garbage collection, and that this was merely a clumsy choice of words, rather than actually insulting these other Python implementations. >> Here's one example: reference cycles. When do they get detected? > > Orphan cycle _detection_ is orthogonal to this proposal. As cycles are always a symptom of a design error I personally have little interest in them other than some facility to detect/debug and eliminate them from applications by design (another topic). > No, they're not always a symptom of a design error. >> Taking a really simple situation: >> >> class Foo: >> def __init__(self): >> self.self = self >> print("Creating a Foo") >> def __del__(self): >> print("Disposing of a Foo") >> >> foo = Foo() >> foo = 1 >> >> When do you expect __del__ to be called? > > At the point of (re)assignment: "foo = 1" > > >> How do you implement this efficiently and reliably? > > CPython already has a reliable and reasonably efficient implementation based on reference counting. > http://catb.org/jargon/html/koans.html Just before reassignment, there are two references to that object: one from the name "foo", and another from its own "self" attribute. After reassignment, there is one reference, from its own "self" attribute. The reference count will never drop to zero. I suggest you read up on why other forms of garbage collection exist, rather than simply demanding that Python behave in a particular way. ChrisA From ooomzay at gmail.com Wed Feb 28 18:53:01 2018 From: ooomzay at gmail.com (ooomzay at gmail.com) Date: Wed, 28 Feb 2018 15:53:01 -0800 (PST) Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: <6ad254cb-7318-4e24-8515-cedf9ab9feb6@googlegroups.com> References: <6ad254cb-7318-4e24-8515-cedf9ab9feb6@googlegroups.com> Message-ID: On Wednesday, February 28, 2018 at 11:45:24 PM UTC, ooo... at gmail.com wrote: > On Wednesday, February 28, 2018 at 11:02:17 PM UTC, Chris Angelico wrote: > > On Thu, Mar 1, 2018 at 9:51 AM, ooomzay wrote: > > [snip] > > Taking a really simple situation: > > > > class Foo: > > def __init__(self): > > self.self = self > > print("Creating a Foo") > > def __del__(self): > > print("Disposing of a Foo") > > > > foo = Foo() > > foo = 1 > > > > When do you expect __del__ to be called? > > At the point of (re)assignment: "foo = 1" Oh... I now see there is a (non-weak) self reference in there. So in this case it would be orphaned. It is a design error and should be recoded. I don't care how it is detected for current purposes. From steve+comp.lang.python at pearwood.info Wed Feb 28 18:58:21 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 28 Feb 2018 23:58:21 +0000 (UTC) Subject: Drinking from the fire-hose Message-ID: Hey Chris, regretting writing that "Statement-Local Name Bindings" PEP yet? :-) I know *I'm* regretting that you wrote it, because this is the sort of language feature that has the potential to make Python significantly ugly and worse, and I'd like to have my say, but there's no way I have time to troll through 50+ and counting (in just 25 hours!) posts. -- Steve From tkadm30 at yandex.com Wed Feb 28 19:01:07 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 28 Feb 2018 19:01:07 -0500 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> Message-ID: <97b607ce-d86f-54e7-bfe0-ee43fadba4aa@yandex.com> What do rats find rewarding in play fighting? Le 2018-02-28 ? 18:29, Chris Angelico a ?crit?: > On Thu, Mar 1, 2018 at 10:23 AM, Steven D'Aprano > wrote: >> I'd go further... what gave you the idea that ANYONE is an expert on >> ultrasonic activation of neural pathways? > Definitely tinfoil hat material here; expertise is completely immaterial. > > ChrisA -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From rosuav at gmail.com Wed Feb 28 19:06:35 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 11:06:35 +1100 Subject: Drinking from the fire-hose In-Reply-To: References: Message-ID: On Thu, Mar 1, 2018 at 10:58 AM, Steven D'Aprano wrote: > Hey Chris, regretting writing that "Statement-Local Name Bindings" PEP > yet? :-) Haha, nah. I'm okay with the fire hose. > I know *I'm* regretting that you wrote it, because this is the sort of > language feature that has the potential to make Python significantly ugly > and worse, and I'd like to have my say, but there's no way I have time to > troll through 50+ and counting (in just 25 hours!) posts. But the proposals keep coming up every now and then. This way, if the PEP gets rejected, it'll be easy to point people to it instead of having to rehash the arguments every time. Maybe flip through the PEP and make comments on that, regardless of previous posts? I'm not unaccustomed to the same topics coming up multiple times during PEP firehosings, and it doesn't much bother me. ChrisA From steve+comp.lang.python at pearwood.info Wed Feb 28 19:14:32 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 1 Mar 2018 00:14:32 +0000 (UTC) Subject: RFC: Proposal: Deterministic Object Destruction References: <6ad254cb-7318-4e24-8515-cedf9ab9feb6@googlegroups.com> Message-ID: On Wed, 28 Feb 2018 15:44:45 -0800, ooomzay wrote: >> Here's one example: reference cycles. When do they get detected? > > Orphan cycle _detection_ is orthogonal to this proposal. It certainly is not. Dealing with cycles is why most of the world has moved on from reference counters. (Or at least, from using reference counters *alone*.) > As cycles are always a symptom of a design error "Always"? So, because you personally don't like the with-statement (possibly because it is too clear, understandable, explicit and deterministic for your tastes?), you want to mandate that everyone else use the clumsy work- around of weak references to avoid an artificial and pointless "design error" of a reference cycle. Great. I'm loving this RFC more and more. Not. Hint: the real world is *full* of cycles. Many problems have to deal with cycles. Having to manage those cycles by hand is *awful*, and completely unnecessary given that garbage collectors exist that can manage them for you. > I personally have little interest in > them other than some facility to detect/debug and eliminate them from > applications by design (another topic). And for your next tricks, you'll solve the political problems of the Middle East without hurting anyone, and then colonise Mars on a budget of ten thousand dollars. -- Steve From rosuav at gmail.com Wed Feb 28 19:19:18 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 11:19:18 +1100 Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: References: Message-ID: On Thu, Mar 1, 2018 at 10:48 AM, Steven D'Aprano wrote: > On Wed, 28 Feb 2018 14:51:09 -0800, ooomzay wrote: >> Motivation >> ========== >> >> To make the elegance (simplicity, generality, and robustness) of the >> "Resource Acquisition Is Initialization" (RAII) idiom available to all >> python applications and libraries. > > I've never yet seen an explanation of RAII that makes the least amount of > sense. I wouldn't call it either elegant or simple. More like "confusing > and badly named". Badly named, yes, but it's not too confusing. RAII is just a fancy name for "when I construct a file object, you open the file, and when the file object is disposed of, you close the file for me". Which is exactly how Python works, with the exception that RAII depends on prompt object disposal (hence the OP's demands). In CPython, you can happily write this: def save_file(fn): f = open(fn, "w") print(data, file=f) print(more_data, file=f) save_file(...) The file object gets disposed of as the function terminates, and the underlying file handle gets closed. That's RAII. The only problem is that this doesn't work reliably with reference counting (the C++ way to do it is to have a single automatic object, no refcounting, no pointers, nothing), and definitely doesn't work with most non-refcount garbage collection (mark-and-sweep, arena, etc). >> Backwards Compatibility >> ======================= >> >> This proposal preserves 100% backwards compatibility. > > So long as you're not using IronPython, Jython, or some alternate builds > of PyPy. > Hmm, I think actually it does; if all Python implementations magically became compliant with the OP's demands, they'd be fully backward-compatible. This is a tightening of requirements on the language. It's still not a good idea though. :) ChrisA From python at mrabarnett.plus.com Wed Feb 28 19:45:56 2018 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 1 Mar 2018 00:45:56 +0000 Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: References: <6ad254cb-7318-4e24-8515-cedf9ab9feb6@googlegroups.com> Message-ID: <02950bf7-23ce-a703-7546-54db5ddb41a2@mrabarnett.plus.com> On 2018-03-01 00:14, Steven D'Aprano wrote: > On Wed, 28 Feb 2018 15:44:45 -0800, ooomzay wrote: > >>> Here's one example: reference cycles. When do they get detected? >> >> Orphan cycle _detection_ is orthogonal to this proposal. > > It certainly is not. Dealing with cycles is why most of the world has > moved on from reference counters. (Or at least, from using reference > counters *alone*.) > > >> As cycles are always a symptom of a design error > > "Always"? > > So, because you personally don't like the with-statement (possibly > because it is too clear, understandable, explicit and deterministic for > your tastes?), you want to mandate that everyone else use the clumsy work- > around of weak references to avoid an artificial and pointless "design > error" of a reference cycle. > > Great. I'm loving this RFC more and more. Not. > > Hint: the real world is *full* of cycles. Many problems have to deal with > cycles. Having to manage those cycles by hand is *awful*, and completely > unnecessary given that garbage collectors exist that can manage them for > you. > > >> I personally have little interest in >> them other than some facility to detect/debug and eliminate them from >> applications by design (another topic). > > And for your next tricks, you'll solve the political problems of the > Middle East without hurting anyone, and then colonise Mars on a budget of > ten thousand dollars. > Or, if you want something a little more challenging, work on Brexit! :-) From rosuav at gmail.com Wed Feb 28 20:32:00 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 12:32:00 +1100 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> Message-ID: On Thu, Mar 1, 2018 at 10:51 AM, Steven D'Aprano wrote: > On Thu, 01 Mar 2018 08:54:52 +1100, Chris Angelico wrote: > >> But this part sounds like prime quality tinfoil hat material: >> >>> Tell me how exactly ultrasonic side channels may activate remotely >>> specific neural pathways implicated in aggressivity and how to block >>> theses specific side channels from neuromodulating human behavior. > > > I wouldn't entirely rule it out. (On the other hand, I wouldn't give it > much credence in the absence of solid evidence either.) I'm too lazy to > google it right now, but I recall that there's definitely evidence for > inaudible *subsonic* frequencies being able to affect human emotions, > mostly to induce feelings of disquiet and nameless dread. Yeah, it might well be possible, but that doesn't change the fact that the greatest interest will come from the same sort of people who turn their PCs' screens to the wall to prevent them from spying during the night, believe that contrails are mind control tools, and wear foil hats to prevent the government from reading their thoughts. ChrisA From ned at nedbatchelder.com Wed Feb 28 20:33:32 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 28 Feb 2018 20:33:32 -0500 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: <97b607ce-d86f-54e7-bfe0-ee43fadba4aa@yandex.com> References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <97b607ce-d86f-54e7-bfe0-ee43fadba4aa@yandex.com> Message-ID: <81d981eb-40b4-8028-fa36-2e48594d75aa@nedbatchelder.com> On 2/28/18 7:01 PM, Etienne Robillard wrote: > > ?What do rats find rewarding in play fighting? This is well outside the topics for this list. --Ned. > > > Le 2018-02-28 ? 18:29, Chris Angelico a ?crit?: >> On Thu, Mar 1, 2018 at 10:23 AM, Steven D'Aprano >> wrote: >>> I'd go further... what gave you the idea that ANYONE is an expert on >>> ultrasonic activation of neural pathways? >> Definitely tinfoil hat material here; expertise is completely >> immaterial. >> >> ChrisA > From python.list at tim.thechases.com Wed Feb 28 20:35:35 2018 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 28 Feb 2018 19:35:35 -0600 Subject: csv module and NULL data byte In-Reply-To: References: Message-ID: <20180228193535.144393fb@bigbox.christie.dr> While inelegant, I've "solved" this with a wrapper/generator f = file(fname, ?) g = (line.replace('\0', '') for line in f) reader = csv.reader(g, ?) for row in reader: process(row) My actual use at $DAYJOB cleans out a few other things too, particularly non-breaking spaces coming from client data that .strip() doesn't catch in Py2.x ("hello\xa0".strip()) -tkc On 2018-02-28 23:40, John Pote wrote: > I have a csv data file that may become corrupted (already happened) > resulting in a NULL byte appearing in the file. The NULL byte > causes an _csv.Error exception. > > I'd rather like the csv reader to return csv lines as best it can > and subsequent processing of each comma separated field deal with > illegal bytes. That way as many lines from the file may be > processed and the corrupted ones simply dumped. > > Is there a way of getting the csv reader to accept all 256 possible > bytes. (with \r,\n and ',' bytes delimiting lines and fields). > > My test code is, > > ??? with open( fname, 'rt', encoding='iso-8859-1' ) as csvfile: > ??? ??? csvreader = csv.reader(csvfile, delimiter=',', > quoting=csv.QUOTE_NONE, strict=False ) > ??? ??? ??? data = list( csvreader ) > ??? ?? ?? ? for ln in data: > ??? ??? ??? ??? print( ln ) > > Result > > >>python36 csvTest.py > Traceback (most recent call last): > ? File "csvTest.py", line 22, in > ??? data = list( csvreader ) > _csv.Error: line contains NULL byte > > strict=False or True makes no difference. > > Help appreciated, > > John > > -- > https://mail.python.org/mailman/listinfo/python-list From rantingrickjohnson at gmail.com Wed Feb 28 21:23:12 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Wed, 28 Feb 2018 18:23:12 -0800 (PST) Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> Message-ID: On Wednesday, February 28, 2018 at 4:34:11 PM UTC-6, Etienne Robillard wrote: > A great number of studies have shown that ultrasonic > neuromodulation of the central nervous system can be > exploited via brain-computer interfaces... It is cutting > edge science however, and my knowledge on techniques for > activating human behavior by remote means is limited. I am > only an amateur scientist/programmer with no experience > whatsoever in Android or neuroscience. I doubt anyone in this group will have knowledge in such a high-tech experimental/theoretical field such as "remote behavior modification"? And even if someone did, i'm sure they'd wise enough to keep quiet about it. Because if such technology exists today, then it would have military applications, and thus, be highly classified. And, just as a thought experiment, if we imagined that -- somehow -- you attained this specialized knowledge, and you were _not_ already employed by one of the military labs -- oh boy! -- you would then become a threat to national security, and be hunted by every spy agency on the face of this planet. With that in mind, i think you're in waaaaay over your head. And perhaps you should apply a reversed transformation to your current vector path. > However, the possibility of ultrasonic side channels to be > exploited by a third-party remotely without the user > consent seem highly likely. I want to develop Python tools > to provide experimental evidences that Android may exploit > ultrasonic side channels to stimulate or impair the > auditory cortex of someone. That sounds malicious to me. Although, considering the abysmal state of music these days... sometimes i wish my smart phone _would_ impair my auditory cortex. > In other words, I plan to disclose the ability of the US > government to use mobile devices as psychoenergetics (non- > lethal) bioweapons. Why take the chance? Ditch the phone all together and live free! Who cares about the lemmings anyway. The worms have already ate into their brains. Even if you provide evidence proving your claims to be true, people (*AHEM*, "Sheeple") are so wedded to their smart phones that even *IF* smart phones caused chronic explosive diarrhea, i seriously doubt anyone would put their phone down for more than five minutes. We're talking about OCD on a _galactic_ scale here, my friend. In fact, smart phone users have become so OCD about their phones, that soon they plan to borrow the NRA's slogan (slightly modified, of course): 'From my *COLD*. *DEAD*. *BROWN*. *HAND*." From gheskett at shentel.net Wed Feb 28 21:30:07 2018 From: gheskett at shentel.net (Gene Heskett) Date: Wed, 28 Feb 2018 21:30:07 -0500 Subject: Problem: Need galileo running on debian wheezy Message-ID: <201802282130.07622.gheskett@shentel.net> I know its supposed to be in the debian stretch repo's. I've been told to get a fitbit, but they don't support linux of any flavor, and that leaves galileo as the possible solution? So how should I proceed since the only stretch machine I have ATM is an arm64, aka a rock64. -- Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From rantingrickjohnson at gmail.com Wed Feb 28 21:46:05 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Wed, 28 Feb 2018 18:46:05 -0800 (PST) Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: References: Message-ID: <20c727d6-e61a-42b1-90c7-c529fe6f67c5@googlegroups.com> On Wednesday, February 28, 2018 at 5:02:17 PM UTC-6, Chris Angelico wrote: > Here's one example: reference cycles. When do they get detected? > Taking a really simple situation: > > class Foo: > def __init__(self): > self.self = self *shudders* Can you provide a real world example in which you need an object which circularly references _itself_? This looks like a highly contrived example used to (1) merely win the argument, and (2) Bump fib() up one position from it's current position as "the worst introductory example of how to write a function in the history of programming tutorials" From rantingrickjohnson at gmail.com Wed Feb 28 21:51:33 2018 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Wed, 28 Feb 2018 18:51:33 -0800 (PST) Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: References: Message-ID: <0e68a121-585e-4670-a864-e0184fed34a2@googlegroups.com> On Wednesday, February 28, 2018 at 5:50:53 PM UTC-6, Steven D'Aprano wrote: > On Wed, 28 Feb 2018 14:51:09 -0800, ooomzay wrote: > > > > [...] > > > > Specification > > ============= > > > > When the last reference to an object goes out of scope the > > intepreter must synchronously, in the thread that releases > > the last reference, invoke the object's __del__() method > > and then free the memory occupied by that object. > > What happens if the __del__ method recreates a reference to > the object? So make instantation of the object in its __del__ method an illegal op! From grant.b.edwards at gmail.com Wed Feb 28 21:51:44 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 1 Mar 2018 02:51:44 +0000 (UTC) Subject: Detection of ultrasonic side channels in mobile devices with Python? References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> Message-ID: On 2018-03-01, Dennis Lee Bieber wrote: > The bandwidth normally used for voice grade telephone traffic is closer > to 6kHz (say 300Hz to 6.3kHz) Wow, that's pretty high -- where was that? Back when I was designing telephony electronics in US in the late 80's, POTS bandwidth was 3KHz: 300-3.3K. That same bandpass spec was used for the audio stages in the first couple generations of AMPS cellular mobile phones. The Codecs used in digital telephony (e.g. a T1 trunk) used 7-bit converts with 8KHz sampling, which means a cutoff frequency below 4KHz. -- Grant From tjreedy at udel.edu Wed Feb 28 21:52:08 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 28 Feb 2018 21:52:08 -0500 Subject: Detection of ultrasonic side channels in mobile devices with Python? In-Reply-To: References: <0314e909-472e-1885-c6a6-4419a3609b8d@yandex.com> <56902f04-bf33-b61e-4762-54eec2625190@nedbatchelder.com> Message-ID: Instead of berating us for not answering an extremely obscure question, you should have use Ms SearchBox first, like I did to determine whether these are real subjects. On 2/28/2018 5:31 PM, Etienne Robillard wrote: > If ultrasonic side channels are a threat to privacy, it is because smart > phones were designed to exploit this precise feature. (Near) Ultrasonic beacons are apparently a real and rather sneaky tracking mechanism. https://www.schneier.com/blog/archives/2017/05/using_ultrasoni.html This is actually about using near-ultrasonic, 18-20 KHtz, which can be produced by some TVs and other off-the-shelf gear and detected by phone microphones. Read or ask *them* about what android/Dalvik system calls they used to access phone microphones. Investigate for yourself whether any Python implementations for Android give you such access. > A great number of studies have shown that ultrasonic neuromodulation of > the central nervous system can be exploited via brain-computer > interfaces... https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4981830/ is a review article about clinical use for brain and nerve stimulation in patients. I am rather sure that they are talking about *real* ultrasound, well away from the audio range. So this seems to be a completely separate topic. -- Terry Jan Reedy From python.list at tim.thechases.com Wed Feb 28 21:52:59 2018 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 28 Feb 2018 20:52:59 -0600 Subject: csv module and NULL data byte In-Reply-To: References: Message-ID: <20180228205259.39594497@bigbox.christie.dr> On 2018-02-28 21:38, Dennis Lee Bieber wrote: > > ??? with open( fname, 'rt', encoding='iso-8859-1' ) as csvfile: > > Pardon? Has the CSV module changed in the last year or so? > > Last time I read the documentation, it was recommended that > the file be opened in BINARY mode ("rb"). It recommends binary mode, but seems to largely work fine with text/ascii mode or even arbitrary iterables. I've not seen the rationale behind the binary recommendation, but in 10+ years of using the csv module, I've not found any issues in using text/ascii mode that were solved by switching to using binary mode. -tkc From tjreedy at udel.edu Wed Feb 28 21:57:43 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 28 Feb 2018 21:57:43 -0500 Subject: csv module and NULL data byte In-Reply-To: <20180228193535.144393fb@bigbox.christie.dr> References: <20180228193535.144393fb@bigbox.christie.dr> Message-ID: On 2/28/2018 8:35 PM, Tim Chase wrote: > While inelegant, I've "solved" this with a wrapper/generator > > f = file(fname, ?) > g = (line.replace('\0', '') for line in f) > reader = csv.reader(g, ?) > for row in reader: > process(row) I think this is elegant in that is cleans the input stream independently of the consumer. It is easier than what I was going to suggest, which is to read the cvs source for the line that raises the exception and modify the code until it works with \0 in the stream. -- Terry Jan Reedy From rosuav at gmail.com Wed Feb 28 22:00:18 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Mar 2018 14:00:18 +1100 Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: <20c727d6-e61a-42b1-90c7-c529fe6f67c5@googlegroups.com> References: <20c727d6-e61a-42b1-90c7-c529fe6f67c5@googlegroups.com> Message-ID: On Thu, Mar 1, 2018 at 1:46 PM, Rick Johnson wrote: > On Wednesday, February 28, 2018 at 5:02:17 PM UTC-6, Chris Angelico wrote: > >> Here's one example: reference cycles. When do they get detected? >> Taking a really simple situation: >> >> class Foo: >> def __init__(self): >> self.self = self > > *shudders* > > Can you provide a real world example in which you need an > object which circularly references _itself_? This looks like > a highly contrived example used to (1) merely win the > argument, and (2) Bump fib() up one position from it's > current position as "the worst introductory example of how > to write a function in the history of programming tutorials" Not off hand, but I can provide an EXTREMELY real-world example of a fairly tight loop: exceptions. An exception has a reference to the local variables it came from, and those locals may well include the exception itself: try: 1/0 except Exception as e: print(e) The ZeroDivisionError has a reference to the locals, and 'e' in the locals refers to that very exception object. ChrisA From dvl at psu.edu Wed Feb 28 23:00:00 2018 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Wed, 28 Feb 2018 23:00:00 -0500 Subject: RFC: Proposal: Deterministic Object Destruction In-Reply-To: mailman.1314.1519872903.2834.python-list@python.org References: Message-ID: <1519876799l.9175092l.0l@psu.edu> On Wed, Feb 28, 2018, Rick Johnson wrote: > On Wednesday, February 28, 2018 at 5:02:17 PM UTC-6, Chris Angelico wrote: > >> Here's one example: reference cycles. When do they get detected? >> Taking a really simple situation: >> >> class Foo: >> def __init__(self): >> self.self = self > >*shudders* > >Can you provide a real world example in which you need an >object which circularly references _itself_? This looks like >a highly contrived example used to (1) merely win the >argument, and (2) Bump fib() up one position from it's >current position as "the worst introductory example of how >to write a function in the history of programming tutorials" If you want something that looks like a real world example, consider the very common doubly-linked list: [ 1 ] <---> [ 2 ] <---> [ 3 ] <--.....--> [ N ] This is chock-full of reference cycles, everywhere you see a <--> Then you have a wrapper List object that has a head referencing the 1 node and a tail referencing the N node. And you decide you don't need that List any more. You could: 1) Take the time the code to carefully delete every single node in this linked list, taking the time to set all of the internal references to None, or 2) Just discard your List object, and let the GC take care of the rest. Note that removing the list alone does not clear any of the reference counts in the picture above, because those internal <--> references are all still there. You either have to manually remove all the references, or rely on the garbage collector. Oh, and for those who claim that reference counting is 'efficient', do bear in mind that every single assignment statement increases the reference count of one value and possibly reduces the reference count of another. So down in the machine or byte-code world, every assignment operation is really two arithmetic computations and three assignments, which takes 5 times as many operations as not doing any reference counting at all. So if you have a program that can get by leaving unused memory allocated, and can still terminate the program before all the memory is claimed, your reference count system is far less efficient than relying on a garbage collector that happens to not get activated before your program ends. Roger Christman Pennsylvania State University From steve+comp.lang.python at pearwood.info Wed Feb 28 23:11:19 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 1 Mar 2018 04:11:19 +0000 (UTC) Subject: RFC: Proposal: Deterministic Object Destruction References: <0e68a121-585e-4670-a864-e0184fed34a2@googlegroups.com> Message-ID: On Wed, 28 Feb 2018 18:51:33 -0800, Rick Johnson wrote: >> What happens if the __del__ method recreates a reference to the object? > > So make instantation of the object in its __del__ method an illegal op! Why should it be illegal? And could that even be enforced in Python? Is this one of those things where we say "In my ignorance and naivety, I am unable to imagine why this could be useful after an entire two seconds of thought, so therefore nobody else could *possibly* find it useful"? Oh, and by the way, the object is already instantiated. The usual term here is *resurrection*: the object had no references to it and so was all but dead the `__del__` method is called, and it does something which ends up creating a new reference to the object and bringing it back to life. While it's true that object resurrection can play havoc with certain (buggy?) compilers[1] and makes the life of the garbage collector more difficult, there are uses for it: https://stackoverflow.com/questions/3680281/usages-of-object-resurrection It is not something I would want to use all the time, but for exceptional circumstances, object resurrection can be useful. [1] I'm lead to believe that early versions of Java would dump core under some circumstances when objects were resurrected. -- Steve From steve+comp.lang.python at pearwood.info Wed Feb 28 23:23:44 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 1 Mar 2018 04:23:44 +0000 (UTC) Subject: RFC: Proposal: Deterministic Object Destruction References: <20c727d6-e61a-42b1-90c7-c529fe6f67c5@googlegroups.com> Message-ID: On Wed, 28 Feb 2018 18:46:05 -0800, Rick Johnson wrote: > On Wednesday, February 28, 2018 at 5:02:17 PM UTC-6, Chris Angelico > wrote: > >> Here's one example: reference cycles. When do they get detected? Taking >> a really simple situation: >> >> class Foo: >> def __init__(self): >> self.self = self > > *shudders* > > Can you provide a real world example in which you need an object which > circularly references _itself_? This looks like a highly contrived > example used to (1) merely win the argument, and (2) Bump fib() up one > position from it's current position as "the worst introductory example > of how to write a function in the history of programming tutorials" Of course it is a contrived example: it is close to the simplest possible example of a reference cycle. Its not intended as an example of useful code. (Useful or not, your interpreter better not crash when faced with a cycle like this.) But of course there are uses for objects to hold a reference to themselves. If you're writing a general purpose collection (say, a list or dict) then you better be able to cope with the case that somebody puts your list in itself: L = [] L.append(L) either directly, as above, or indirectly. And that's the point really: cycles don't just occur when an object holds a reference to itself. They occur when one object holds a reference to another object, which holds a reference to a third, which holds a reference to a fourth (and so on...), which eventually holds a reference to the original again. Cycles are useful. For example, you might think of a window which holds a reference to a control, which in turn needs a reference to its owning window. Or a network which contains loops, say a network of streets, or networks of disease transmission (I caught the stomach bug from Bob, who caught it from Sally, who caught it from Georgina, who originally caught it from me the *first* time I had it). It shouldn't take much imagination to think of cycles in real life which you may want to model in a program. https://en.wikipedia.org/wiki/I'm_My_Own_Grandpa -- Steve