From bengt.tornq at gmail.com Wed Aug 1 00:52:39 2018 From: bengt.tornq at gmail.com (bengt.tornq at gmail.com) Date: Tue, 31 Jul 2018 21:52:39 -0700 (PDT) Subject: How to start gnuradio In-Reply-To: References: Message-ID: <5df9b0f3-075b-4ba1-84be-402a973a9f19@googlegroups.com> After some research I found out that "sudo apt-get install python-numpy" solved the problem. Can anyone clarify how python-numpy solves the problem? From rzzzwilson at gmail.com Wed Aug 1 01:23:47 2018 From: rzzzwilson at gmail.com (Ross Wilson) Date: Wed, 1 Aug 2018 12:23:47 +0700 Subject: How to start gnuradio In-Reply-To: <5df9b0f3-075b-4ba1-84be-402a973a9f19@googlegroups.com> References: <5df9b0f3-075b-4ba1-84be-402a973a9f19@googlegroups.com> Message-ID: I had a paddle through the manual at https://www.gnuradio.org/doc/doxygen/page_python_blocks.html and apparently some DSP operations use numpy. Ross On Wed, 1 Aug 2018 at 11:56 wrote: > > > After some research I found out that "sudo apt-get install python-numpy" > solved the problem. > > Can anyone clarify how python-numpy solves the problem? > -- > https://mail.python.org/mailman/listinfo/python-list > From ben+python at benfinney.id.au Wed Aug 1 01:58:32 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 01 Aug 2018 15:58:32 +1000 Subject: How to start gnuradio References: Message-ID: <861sbibpbr.fsf@benfinney.id.au> bengt.tornq at gmail.com writes: > My gnuradio program does not start in my Mint 17.3 installation. Summary: I believe this is a bug in the package on Mint. (The bug may be inherited from elsewhere, too: maybe in the Debian package, maybe in the PyPI package. That would need more diagnosis to determine.) > When I invoke the command "gnuradio-companion" I get the following message: > > ----------------------------- > Cannot import gnuradio. That looks like a bug, indeed. Because you later reported that installing another Python package brings correct behaviour, this strongly indicates that the gnuradio package does not correctly declare its dependencies. In other words: The 'gnuradio-companion' program does not work without an additional package installed, so that is a dependency which must be declared for the 'gnuradio' package. > Grateful for hints on how to find out the reason and to solve the > problem. I think you have enough information to write a bug report to the Mint bug tracking system for this. -- \ ?The good thing about science is that it's true whether or not | `\ you believe in it.? ?Neil deGrasse Tyson, 2011-02-04 | _o__) | Ben Finney From bengt.tornq at gmail.com Wed Aug 1 04:04:23 2018 From: bengt.tornq at gmail.com (bengt.tornq at gmail.com) Date: Wed, 1 Aug 2018 01:04:23 -0700 (PDT) Subject: How to start gnuradio In-Reply-To: References: <861sbibpbr.fsf@benfinney.id.au> Message-ID: Thank you all for your kind explanations. From robin at reportlab.com Wed Aug 1 04:36:16 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 1 Aug 2018 09:36:16 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> Message-ID: On 31/07/2018 16:52, Chris Angelico wrote: > On Wed, Aug 1, 2018 at 1:28 AM, MRAB wrote: >> On 2018-07-31 08:40, Robin Becker wrote: >>> >>> A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings ............. >> The warning looks wrong to be. >> >> In Python 2, u'a' and b'a' would be treated as the same key, but in Python 3 >> they are distinct and can co-exist. >> >> Something for Python's bug tracker, I think! > > It's a warning specifically requested by the -b option. The two keys > in question have the same hash, which means they have to be compared > directly; they will compare unequal, but because of the -b flag, the > comparison triggers a warning. If that warning is spurious, *don't use > the -b option*. > > ChrisA > I looked at the language documentation for 3.7 mappings > These represent finite sets of objects indexed by nearly arbitrary values. The only types of values not acceptable as keys are values containing lists or dictionaries or other mutable types that are compared by value rather than by object identity, the reason being that the efficient implementation of dictionaries requires a key?s hash value to remain constant. Numeric types used for keys obey the normal rules for numeric comparison: if two numbers compare equal (e.g., 1 and 1.0) then they can be used interchangeably to index the same dictionary entry. it says explicitly that numeric keys will use numeric comparison, but no mention is made of strings/bytes etc etc and there's an implication that object identity is used rather than comparison. In python 3.x b'a' is not the same as 'a' either the documentation is lacking some reference to strings/bytes or the warning is wrong. Using the excuse that normal comparison is being used seems a bit feeble. It would clearly improve speed if object identity were to be used, but I suppose that's not the case for other reasons. -- Robin Becker From rosuav at gmail.com Wed Aug 1 04:52:02 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 1 Aug 2018 18:52:02 +1000 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> Message-ID: On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote: > On 31/07/2018 16:52, Chris Angelico wrote: >> >> On Wed, Aug 1, 2018 at 1:28 AM, MRAB wrote: >>> >>> On 2018-07-31 08:40, Robin Becker wrote: >>>> >>>> >>>> A bitbucket user complains that python 3.6.6 with -Wall -b prints >>>> warnings > > ............. >>> >>> The warning looks wrong to be. >>> >>> In Python 2, u'a' and b'a' would be treated as the same key, but in >>> Python 3 >>> they are distinct and can co-exist. >>> >>> Something for Python's bug tracker, I think! >> >> >> It's a warning specifically requested by the -b option. The two keys >> in question have the same hash, which means they have to be compared >> directly; they will compare unequal, but because of the -b flag, the >> comparison triggers a warning. If that warning is spurious, *don't use >> the -b option*. >> >> ChrisA >> > > I looked at the language documentation for 3.7 mappings > >> These represent finite sets of objects indexed by nearly arbitrary values. >> The only types of values not acceptable as keys are values containing lists >> or dictionaries or other mutable types that are compared by value rather >> than by object identity, the reason being that the efficient implementation >> of dictionaries requires a key?s hash value to remain constant. Numeric >> types used for keys obey the normal rules for numeric comparison: if two >> numbers compare equal (e.g., 1 and 1.0) then they can be used >> interchangeably to index the same dictionary entry. > > > > > it says explicitly that numeric keys will use numeric comparison, but no > mention is made of strings/bytes etc etc and there's an implication that > object identity is used rather than comparison. In python 3.x b'a' is not > the same as 'a' either the documentation is lacking some reference to > strings/bytes or the warning is wrong. Using the excuse that normal > comparison is being used seems a bit feeble. It would clearly improve speed > if object identity were to be used, but I suppose that's not the case for > other reasons. > Technically, the comparison used is: a is b or a == b in other words, identity will match, but mainly, equality is used. The identity check improves performance in many common cases, and also avoids pathological cases involving float("nan"), but in general discussion, it's assumed that value rather than identity is the comparison used. ChrisA From robin at reportlab.com Wed Aug 1 05:43:31 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 1 Aug 2018 10:43:31 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> Message-ID: <35155789-0d99-a86d-232a-7fb0d23509f6@chamonix.reportlab.co.uk> On 01/08/2018 09:52, Chris Angelico wrote: > On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote: >> On 31/07/2018 16:52, Chris Angelico wrote: >.......... >> >> it says explicitly that numeric keys will use numeric comparison, but no ......... >> > > Technically, the comparison used is: > > a is b or a == b > > in other words, identity will match, but mainly, equality is used. The > identity check improves performance in many common cases, and also > avoids pathological cases involving float("nan"), but in general > discussion, it's assumed that value rather than identity is the > comparison used. > > ChrisA > If the testing were done in that order then no comparison warning would occur as inspection shows 'a' is not b'a' > C:\code\hg-repos\reportlab\tmp>\python37\python.exe -b -Wall > Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> 'a' is b'a' > False >>>> so presumably the testing goes something like a is b or (a==b if comparable(a,b) else False) even so I still think errors/warnings created internally by implementers should not be exposed. It's not a big deal. I'm a bit surprised that we don't have a mapping which uses only identity as that would be faster. -- Robin Becker From rosuav at gmail.com Wed Aug 1 05:59:07 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 1 Aug 2018 19:59:07 +1000 Subject: Are dicts supposed to raise comparison errors In-Reply-To: <35155789-0d99-a86d-232a-7fb0d23509f6@chamonix.reportlab.co.uk> References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <35155789-0d99-a86d-232a-7fb0d23509f6@chamonix.reportlab.co.uk> Message-ID: On Wed, Aug 1, 2018 at 7:43 PM, Robin Becker wrote: > On 01/08/2018 09:52, Chris Angelico wrote: >> >> On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote: >>> >>> On 31/07/2018 16:52, Chris Angelico wrote: >> >> .......... >>> >>> >>> it says explicitly that numeric keys will use numeric comparison, but no > > ......... >>> >>> >> >> Technically, the comparison used is: >> >> a is b or a == b >> >> in other words, identity will match, but mainly, equality is used. The >> identity check improves performance in many common cases, and also >> avoids pathological cases involving float("nan"), but in general >> discussion, it's assumed that value rather than identity is the >> comparison used. >> >> ChrisA >> > > If the testing were done in that order then no comparison warning would > occur as inspection shows 'a' is not b'a' > > > C:\code\hg-repos\reportlab\tmp>\python37\python.exe -b -Wall >> >> Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit >> (AMD64)] on win32 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> >>>>> 'a' is b'a' >> >> False >>>>> >>>>> > > so presumably the testing goes something like > > a is b or (a==b if comparable(a,b) else False) > > even so I still think errors/warnings created internally by implementers > should not be exposed. It's not a big deal. I'm a bit surprised that we > don't have a mapping which uses only identity as that would be faster. > I'm not sure how you're interpreting the word 'or' in my explanation, but it has the same meaning it has in Python code. ChrisA From __peter__ at web.de Wed Aug 1 06:00:54 2018 From: __peter__ at web.de (Peter Otten) Date: Wed, 01 Aug 2018 12:00:54 +0200 Subject: Are dicts supposed to raise comparison errors References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <35155789-0d99-a86d-232a-7fb0d23509f6@chamonix.reportlab.co.uk> Message-ID: Robin Becker wrote: > On 01/08/2018 09:52, Chris Angelico wrote: >> On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote: >>> On 31/07/2018 16:52, Chris Angelico wrote: >>.......... >>> >>> it says explicitly that numeric keys will use numeric comparison, but no > ......... >>> >> >> Technically, the comparison used is: >> >> a is b or a == b >> >> in other words, identity will match, but mainly, equality is used. The >> identity check improves performance in many common cases, and also >> avoids pathological cases involving float("nan"), but in general >> discussion, it's assumed that value rather than identity is the >> comparison used. >> >> ChrisA >> > > If the testing were done in that order then no comparison warning would > occur as inspection shows 'a' is not b'a' Nope, that would be the effect of "and", not "or". >>> "a" is b"b" and "fallback" False >>> "a" is b"b" or "fallback" 'fallback' You seem to be caught in your wrong mental model. I recommend that you let the matter rest for a day or so, and then look at it with a fresh eye. > > C:\code\hg-repos\reportlab\tmp>\python37\python.exe -b -Wall >> Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 >> bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" >> for more information. >>>>> 'a' is b'a' >> False >>>>> > > so presumably the testing goes something like > > a is b or (a==b if comparable(a,b) else False) > > even so I still think errors/warnings created internally by implementers > should not be exposed. It's not a big deal. I'm a bit surprised that we > don't have a mapping which uses only identity as that would be faster. From robin at reportlab.com Wed Aug 1 06:34:01 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 1 Aug 2018 11:34:01 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <35155789-0d99-a86d-232a-7fb0d23509f6@chamonix.reportlab.co.uk> Message-ID: <4de68662-cf00-7176-ac67-6994dc027ee6@chamonix.reportlab.co.uk> .......... > Nope, that would be the effect of "and", not "or". > >>>> "a" is b"b" and "fallback" > False >>>> "a" is b"b" or "fallback" > 'fallback' > > You seem to be caught in your wrong mental model. I recommend that you let > the matter rest for a day or so, and then look at it with a fresh eye. >...... my bad; not worrying enough about real problems -- Robin Becker From tjol at tjol.eu Wed Aug 1 06:40:32 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 1 Aug 2018 12:40:32 +0200 Subject: Confused on git commit tree about Lib/datetime.py In-Reply-To: References: Message-ID: On 01/08/18 05:16, Jeffrey Zhang wrote: > I found a interesting issue when checking the Lib/datetime.py > implementation in python3 > > This patch is introduced by cf86e368ebd17e10f68306ebad314eea31daaa1e [0]. > But if you > check the github page[0], or using git tag --contains, you will find v2.7.x > includes this commit too. > > $ git tag --contains cf86e368ebd17e10f68306ebad314eea31daaa1e > 3.2 > v2.7.10 > v2.7.10rc1 > v2.7.11 > v2.7.11rc1 > ... > > whereas, if you check the v2.7.x code base, nothing is found > > $ git log v2.7.4 -- Lib/datetime.py > > > I guess it maybe a git tool bug, or the commit tree is messed up. Is there > any guys could explain this > situation? > > [0] > https://github.com/python/cpython/commit/cf86e368ebd17e10f68306ebad314eea31daaa1e > There are some crazy merges in the repository history, that are easy to see with gitk (and not so easy to see with github or the command-line porcelain) In this case, it appears that d26b658f1433a28b611906c078f47bc804a63dd1, in the mercurial days, cherry-picked a patch from the 3.2 branch at f8b9dfd9a1a3d611d43e3c3ef8031fed96c8dd25. I don't know why the git tools don't find this by themselves. I suppose this is a quirk of the hg-git transition and the different ways hg and git have of doing things. -- Thomas From rosuav at gmail.com Wed Aug 1 06:49:33 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 1 Aug 2018 20:49:33 +1000 Subject: Confused on git commit tree about Lib/datetime.py In-Reply-To: References: Message-ID: On Wed, Aug 1, 2018 at 8:40 PM, Thomas Jollans wrote: > On 01/08/18 05:16, Jeffrey Zhang wrote: >> >> I found a interesting issue when checking the Lib/datetime.py >> implementation in python3 >> >> This patch is introduced by cf86e368ebd17e10f68306ebad314eea31daaa1e [0]. >> But if you >> check the github page[0], or using git tag --contains, you will find >> v2.7.x >> includes this commit too. >> >> $ git tag --contains cf86e368ebd17e10f68306ebad314eea31daaa1e >> 3.2 >> v2.7.10 >> v2.7.10rc1 >> v2.7.11 >> v2.7.11rc1 >> ... >> >> whereas, if you check the v2.7.x code base, nothing is found >> >> $ git log v2.7.4 -- Lib/datetime.py >> >> >> I guess it maybe a git tool bug, or the commit tree is messed up. Is there >> any guys could explain this >> situation? >> >> [0] >> >> https://github.com/python/cpython/commit/cf86e368ebd17e10f68306ebad314eea31daaa1e >> > There are some crazy merges in the repository history, that are easy to see > with gitk (and not so easy to see with github or the command-line porcelain) > > In this case, it appears that d26b658f1433a28b611906c078f47bc804a63dd1, in > the mercurial days, cherry-picked a patch from the 3.2 branch at > f8b9dfd9a1a3d611d43e3c3ef8031fed96c8dd25. > > I don't know why the git tools don't find this by themselves. I suppose this > is a quirk of the hg-git transition and the different ways hg and git have > of doing things. I'm not sure what you're expecting them to find. They find that the commit in question is part of the 2.7 branch, so every parent of it is also part of the 2.7 branch. They are correctly reporting it. The commit is a bit weird, but the tools are faithfully reporting the weirdness. ChrisA From flebber.crue at gmail.com Wed Aug 1 07:30:51 2018 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Wed, 1 Aug 2018 04:30:51 -0700 (PDT) Subject: Use a function arg in soup Message-ID: <223d4208-79b2-4e21-a6bd-37e5fda960ce@googlegroups.com> Hi. I want to use a function argument as an argument to a bs4 search for attributes. I had this working not as a function # noms = soup.findAll('nomination') # nom_attrs = [] # for attr in soup.nomination.attrs: # nom_attrs.append(attr) But as I wanted to keep finding other elements I thought I would turn it into a generator function. def attrKey(element): for attr in soup.element.attrs: yield attr results in a Nonetype as soup.element.attrs is passed and the attribute isn't substituted. # Attempt 2 def attrKey(element): for attr in "soup.{}.attrs".format(element): yield attr # Attempt 3 def attrKey(element): search_attr = "soup.{}.attrs".format(element) for attr in search_attr: yield attr so I would call it like attrKey('nomination') Any ideas on how the function argument can be used as the search attribute? Thanks Sayth From Richard at Damon-Family.org Wed Aug 1 07:56:11 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Wed, 1 Aug 2018 07:56:11 -0400 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> Message-ID: <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> On 8/1/18 4:36 AM, Robin Becker wrote: > On 31/07/2018 16:52, Chris Angelico wrote: >> On Wed, Aug 1, 2018 at 1:28 AM, MRAB wrote: >>> On 2018-07-31 08:40, Robin Becker wrote: >>>> >>>> A bitbucket user complains that python 3.6.6 with -Wall -b prints >>>> warnings > ............. >>> The warning looks wrong to be. >>> >>> In Python 2, u'a' and b'a' would be treated as the same key, but in >>> Python 3 >>> they are distinct and can co-exist. >>> >>> Something for Python's bug tracker, I think! >> >> It's a warning specifically requested by the -b option. The two keys >> in question have the same hash, which means they have to be compared >> directly; they will compare unequal, but because of the -b flag, the >> comparison triggers a warning. If that warning is spurious, *don't use >> the -b option*. >> >> ChrisA >> > > I looked at the language documentation for 3.7 mappings > >> These represent finite sets of objects indexed by nearly arbitrary >> values. The only types of values not acceptable as keys are values >> containing lists or dictionaries or other mutable types that are >> compared by value rather than by object identity, the reason being >> that the efficient implementation of dictionaries requires a key?s >> hash value to remain constant. Numeric types used for keys obey the >> normal rules for numeric comparison: if two numbers compare equal >> (e.g., 1 and 1.0) then they can be used interchangeably to index the >> same dictionary entry. > > > > it says explicitly that numeric keys will use numeric comparison, but > no mention is made of strings/bytes etc etc and there's an implication > that object identity is used rather than comparison. In python 3.x > b'a' is not the same as 'a' either the documentation is lacking some > reference to strings/bytes or the warning is wrong. Using the excuse > that normal comparison is being used seems a bit feeble. It would > clearly improve speed if object identity were to be used, but I > suppose that's not the case for other reasons. One problem with using identity for strings is that strings that have the same value may not have the same identity. I believe very short strings, like small numbers are cached, so same valued (and typed) short strings will have the same identity, but longer strings are not. It might work for 'a', but might not for 'employee'. -- Richard Damon From rosuav at gmail.com Wed Aug 1 08:07:04 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 1 Aug 2018 22:07:04 +1000 Subject: Are dicts supposed to raise comparison errors In-Reply-To: <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> Message-ID: On Wed, Aug 1, 2018 at 9:56 PM, Richard Damon wrote: >> it says explicitly that numeric keys will use numeric comparison, but >> no mention is made of strings/bytes etc etc and there's an implication >> that object identity is used rather than comparison. In python 3.x >> b'a' is not the same as 'a' either the documentation is lacking some >> reference to strings/bytes or the warning is wrong. Using the excuse >> that normal comparison is being used seems a bit feeble. It would >> clearly improve speed if object identity were to be used, but I >> suppose that's not the case for other reasons. > > One problem with using identity for strings is that strings that have > the same value may not have the same identity. I believe very short > strings, like small numbers are cached, so same valued (and typed) short > strings will have the same identity, but longer strings are not. It > might work for 'a', but might not for 'employee'. Strings and numbers MUST be compared by value, not identity. CPython has a cache for strings used in compilation, but that's about it: >>> s1 = "spam" >>> s2, s3 = "sp", "am" >>> s4 = s2 + s3 >>> s1 is s4, s1 == s4 (False, True) >>> n1 = 42 >>> n2 = n1 * 3 >>> n3 = n2 / 3 >>> n1 is n3, n1 == n3 (False, True) >>> n4 = 123456789 >>> n5 = n4 * 3 >>> n6 = n5 // 3 >>> n4 is n6, n4 == n6 (False, True) The middle example is guaranteed (because n1 is an int, n3 is a float), but the other two could in theory be cached by a compliant Python implementation. I don't know of any that do, though. ChrisA From __peter__ at web.de Wed Aug 1 08:09:29 2018 From: __peter__ at web.de (Peter Otten) Date: Wed, 01 Aug 2018 14:09:29 +0200 Subject: Use a function arg in soup References: <223d4208-79b2-4e21-a6bd-37e5fda960ce@googlegroups.com> Message-ID: Sayth Renshaw wrote: > Hi. > > I want to use a function argument as an argument to a bs4 search for > attributes. > > I had this working not as a function > # noms = soup.findAll('nomination') > # nom_attrs = [] > # for attr in soup.nomination.attrs: > # nom_attrs.append(attr) > But as I wanted to keep finding other elements I thought I would turn it > into a generator function. > > > def attrKey(element): > for attr in soup.element.attrs: > yield attr > > results in a Nonetype as soup.element.attrs is passed and the attribute > isn't substituted. (1) soup.nomination will only give the first , so in the general case soup.find_all("nomination") is the better approach. (2) attrs is a dict, so iterating over it will lose the values. Are you sure you want that? If you use find_all() that already takes a string, so >>> from bs4 import BeautifulSoup as BS >>> soup = BS("FOOBARFOO2") >>> def gen_attrs(soup, element): ... for e in soup.find_all(element): yield e.attrs ... >>> list(gen_attrs(soup, "foo")) [{'b': '2', 'a': '1'}, {'d': '42'}] To access an attribute programatically there's the getattr() builtin; foo.bar is equivalent to getattr(foo, "bar"), so >>> def get_attrs(soup, element): ... return getattr(soup, element).attrs ... >>> get_attrs(soup, "foo") {'b': '2', 'a': '1'} > > # Attempt 2 > def attrKey(element): > for attr in "soup.{}.attrs".format(element): > yield attr > > # Attempt 3 > def attrKey(element): > search_attr = "soup.{}.attrs".format(element) > for attr in search_attr: > yield attr > > > so I would call it like > attrKey('nomination') > > Any ideas on how the function argument can be used as the search > attribute? > > Thanks > > Sayth From robin at reportlab.com Wed Aug 1 09:25:25 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 1 Aug 2018 14:25:25 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> Message-ID: messing with bytes I discover that this doesn't warn with python -b ######################## if __name__=='__main__': class nbytes(bytes): def __eq__(self,other): return bytes.__eq__(self,other) if isinstance(other,bytes) else False def __hash__(self): return bytes.__hash__(self) d={'a':1} d[nbytes(b'a')] = d['a'] print(d) d={nbytes(b'a'):1} d['a'] = d[b'a'] print(d) ######################## > C:\code\hg-repos\reportlab\tmp>\python37\python -b tb1.py > {'a': 1, b'a': 1} > {b'a': 1, 'a': 1} I expected one of the assignments to warn. -- Robin Becker From rosuav at gmail.com Wed Aug 1 09:38:25 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 1 Aug 2018 23:38:25 +1000 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> Message-ID: On Wed, Aug 1, 2018 at 11:25 PM, Robin Becker wrote: > messing with bytes I discover that this doesn't warn with python -b > > ######################## > if __name__=='__main__': > class nbytes(bytes): > def __eq__(self,other): > return bytes.__eq__(self,other) if isinstance(other,bytes) else > False > def __hash__(self): > return bytes.__hash__(self) > d={'a':1} > d[nbytes(b'a')] = d['a'] > print(d) > d={nbytes(b'a'):1} > d['a'] = d[b'a'] > print(d) > ######################## > > >> C:\code\hg-repos\reportlab\tmp>\python37\python -b tb1.py >> {'a': 1, b'a': 1} >> {b'a': 1, 'a': 1} > > > I expected one of the assignments to warn. It's a warning designed to help people port code from Py2 to Py3. It's not meant to catch every possible comparison. Unless you are actually porting Py2 code and are worried that you'll be accidentally comparing bytes and text, just *don't use the -b switch* and there will be no problems. I don't understand what the issue is here. ChrisA From robin at reportlab.com Wed Aug 1 11:07:29 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 1 Aug 2018 16:07:29 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> Message-ID: <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> On 01/08/2018 14:38, Chris Angelico wrote: > t's a warning designed to help people port code from Py2 to Py3. It's > not meant to catch every possible comparison. Unless you are actually > porting Py2 code and are worried that you'll be accidentally comparing > bytes and text, just*don't use the -b switch* and there will be no > problems. > > I don't understand what the issue is here. I don't either, I have never used the -b flag until the issue was raised on bitbucket. If someone is testing a program with reportlab and uses that flag then they get a lot of warnings from this dictionary assignment. Probably the code needs tightening so that we insist on using native strings everywhere; that's quite hard for py2/3 compatible code. -- Robin Becker From p.f.moore at gmail.com Wed Aug 1 11:22:16 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 1 Aug 2018 16:22:16 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On Wed, 1 Aug 2018 at 16:10, Robin Becker wrote: > > On 01/08/2018 14:38, Chris Angelico wrote: > > t's a warning designed to help people port code from Py2 to Py3. It's > > not meant to catch every possible comparison. Unless you are actually > > porting Py2 code and are worried that you'll be accidentally comparing > > bytes and text, just*don't use the -b switch* and there will be no > > problems. > > > > I don't understand what the issue is here. > > I don't either, I have never used the -b flag until the issue was raised on bitbucket. If someone is testing a program with > reportlab and uses that flag then they get a lot of warnings from this dictionary assignment. Probably the code needs tightening > so that we insist on using native strings everywhere; that's quite hard for py2/3 compatible code. They should probably use the warnings module to disable the warning in library code that they don't control, in that case. If they've reported to you that your code produces warnings under -b, your response can quite reasonably be "thanks for the information, we've reviewed our bytes/string handling and can confirm that it's safe, so there's no fixes needed in reportlab". Paul From rosuav at gmail.com Wed Aug 1 12:03:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 2 Aug 2018 02:03:08 +1000 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On Thu, Aug 2, 2018 at 1:22 AM, Paul Moore wrote: > On Wed, 1 Aug 2018 at 16:10, Robin Becker wrote: >> >> On 01/08/2018 14:38, Chris Angelico wrote: >> > t's a warning designed to help people port code from Py2 to Py3. It's >> > not meant to catch every possible comparison. Unless you are actually >> > porting Py2 code and are worried that you'll be accidentally comparing >> > bytes and text, just*don't use the -b switch* and there will be no >> > problems. >> > >> > I don't understand what the issue is here. >> >> I don't either, I have never used the -b flag until the issue was raised on bitbucket. If someone is testing a program with >> reportlab and uses that flag then they get a lot of warnings from this dictionary assignment. Probably the code needs tightening >> so that we insist on using native strings everywhere; that's quite hard for py2/3 compatible code. > > They should probably use the warnings module to disable the warning in > library code that they don't control, in that case. > > If they've reported to you that your code produces warnings under -b, > your response can quite reasonably be "thanks for the information, > we've reviewed our bytes/string handling and can confirm that it's > safe, so there's no fixes needed in reportlab". Yep, agreed. And I suspect that there may be a bit of non-thinking-C-mentality creeping in: "if I can turn on warnings, I should, and any warning is a problem". That simply isn't the case in Python. ChrisA From tjol at tjol.eu Wed Aug 1 12:12:02 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 1 Aug 2018 18:12:02 +0200 Subject: Confused on git commit tree about Lib/datetime.py In-Reply-To: References: Message-ID: <1a1703f0-a7fd-b5e3-5fb7-9007ef94acc1@tjol.eu> On 01/08/18 12:49, Chris Angelico wrote: > On Wed, Aug 1, 2018 at 8:40 PM, Thomas Jollans wrote: >> On 01/08/18 05:16, Jeffrey Zhang wrote: >>> I found a interesting issue when checking the Lib/datetime.py >>> implementation in python3 >>> >>> This patch is introduced by cf86e368ebd17e10f68306ebad314eea31daaa1e [0]. >>> But if you >>> check the github page[0], or using git tag --contains, you will find >>> v2.7.x >>> includes this commit too. >>> >>> $ git tag --contains cf86e368ebd17e10f68306ebad314eea31daaa1e >>> 3.2 >>> v2.7.10 >>> v2.7.10rc1 >>> v2.7.11 >>> v2.7.11rc1 >>> ... >>> >>> whereas, if you check the v2.7.x code base, nothing is found >>> >>> $ git log v2.7.4 -- Lib/datetime.py >>> >>> >>> I guess it maybe a git tool bug, or the commit tree is messed up. Is there >>> any guys could explain this >>> situation? >>> >>> [0] >>> >>> https://github.com/python/cpython/commit/cf86e368ebd17e10f68306ebad314eea31daaa1e >>> >> There are some crazy merges in the repository history, that are easy to see >> with gitk (and not so easy to see with github or the command-line porcelain) >> >> In this case, it appears that d26b658f1433a28b611906c078f47bc804a63dd1, in >> the mercurial days, cherry-picked a patch from the 3.2 branch at >> f8b9dfd9a1a3d611d43e3c3ef8031fed96c8dd25. >> >> I don't know why the git tools don't find this by themselves. I suppose this >> is a quirk of the hg-git transition and the different ways hg and git have >> of doing things. > I'm not sure what you're expecting them to find. They find that the > commit in question is part of the 2.7 branch, so every parent of it is > also part of the 2.7 branch. They are correctly reporting it. The > commit is a bit weird, but the tools are faithfully reporting the > weirdness. > > ChrisA That's not the big I'm confused about. I (like Jeffrey) would have expected git log v2.7.4 -- Lib/datetime.py to show something. But right now I'm not sure that ever shows anything for deleted files, so there's that. Thomas From __peter__ at web.de Wed Aug 1 13:19:55 2018 From: __peter__ at web.de (Peter Otten) Date: Wed, 01 Aug 2018 19:19:55 +0200 Subject: Are dicts supposed to raise comparison errors References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: Paul Moore wrote: > On Wed, 1 Aug 2018 at 16:10, Robin Becker wrote: >> >> On 01/08/2018 14:38, Chris Angelico wrote: >> > t's a warning designed to help people port code from Py2 to Py3. It's >> > not meant to catch every possible comparison. Unless you are actually >> > porting Py2 code and are worried that you'll be accidentally comparing >> > bytes and text, just*don't use the -b switch* and there will be no >> > problems. >> > >> > I don't understand what the issue is here. >> >> I don't either, I have never used the -b flag until the issue was raised >> on bitbucket. If someone is testing a program with reportlab and uses >> that flag then they get a lot of warnings from this dictionary >> assignment. Probably the code needs tightening so that we insist on using >> native strings everywhere; that's quite hard for py2/3 compatible code. > > They should probably use the warnings module to disable the warning in > library code that they don't control, in that case. > > If they've reported to you that your code produces warnings under -b, > your response can quite reasonably be "thanks for the information, > we've reviewed our bytes/string handling and can confirm that it's > safe, so there's no fixes needed in reportlab". > > Paul I've looked into the actual code which has # paraparser.py f = isPy3 and asBytes or asUnicode K = list(known_entities.keys()) for k in K: known_entities[f(k)] = known_entities[k] It looks like known_entities starts out with the default string type, i. e. unicode in py3 and bytes in py2. While in py2 the code has no effect in py3 it adds the corresponding keys of type bytes. However, known_entities is then used in HTMLParser.handle_entity_ref(self, name) which passes the name as unicode in py3. I didn't try, but I would suspect that the module keeps working as expected when you remove the lines quoted above. If I'm correct running the program with the -b flag has at least helped in cleaning up the code in this case. In other cases it might detect sources of bugs, so IMHO it's better to have a close look at and possibly rewrite code that triggers the warning rather than to disable it. From steve+comp.lang.python at pearwood.info Wed Aug 1 13:37:59 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 1 Aug 2018 17:37:59 +0000 (UTC) Subject: Are dicts supposed to raise comparison errors References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On Wed, 01 Aug 2018 16:22:16 +0100, Paul Moore wrote: > On Wed, 1 Aug 2018 at 16:10, Robin Becker wrote: >> >> On 01/08/2018 14:38, Chris Angelico wrote: >> > t's a warning designed to help people port code from Py2 to Py3. It's >> > not meant to catch every possible comparison. Unless you are actually >> > porting Py2 code and are worried that you'll be accidentally >> > comparing bytes and text, just*don't use the -b switch* and there >> > will be no problems. >> > >> > I don't understand what the issue is here. >> >> I don't either, I have never used the -b flag until the issue was >> raised on bitbucket. If someone is testing a program with reportlab and >> uses that flag then they get a lot of warnings from this dictionary >> assignment. Probably the code needs tightening so that we insist on >> using native strings everywhere; that's quite hard for py2/3 compatible >> code. > > They should probably use the warnings module to disable the warning in > library code that they don't control, in that case. > > If they've reported to you that your code produces warnings under -b, > your response can quite reasonably be "thanks for the information, we've > reviewed our bytes/string handling and can confirm that it's safe, so > there's no fixes needed in reportlab". I'm sorry, I don't understand this reasoning. (Perhaps I have missed something.) Robin says his code runs under both Python2 and Python3. He's getting a warning that the behaviour has changed between the two, and there's a dubious comparison being made between bytes and strings. Consequently, there's a very real chance that he has a dicts which have one key in Python 2 but two in Python 3: - in Python 2, b'x' and u'x' are the same key; - in Python 3, b'x' and u'x' are different keys; # Python 2 py> {u'x': 1, b'x': 2} {u'x': 2} #Python 3 py> {u'x': 1, b'x': 2} {b'x': 2, 'x': 1} This means that Robin very likely has subtly or not-so-subtly different behaviour his software depending on which version of Python it runs under. If not an outright bug that silently does the wrong thing. Even if Robin has audited the entire code base and can confidently say today that despite the warning, no such bug has manifested, he cannot possibly be sure that it won't manifest tomorrow. (Not unless the software is frozen and will never be modified.) In another post, Chris says: I suspect that there may be a bit of non-thinking-C-mentality creeping in: "if I can turn on warnings, I should, and any warning is a problem". That simply isn't the case in Python. I strongly disagree. Unless Chris' intention is to say bugs don't matter if they're written in Python, I don't know why one would say that warnings aren't a problem. Every warning is one of three cases: - it reveals an existing actual problem; - it reveals a potential problem which might somebody become an actual problem; - or it is a false positive which (if unfixed) distracts attention and encourages a blas? attitude which could easily lead to problems in the future. Warnings are a code smell. Avoiding warnings is a way of "working clean": https://blog.codinghorror.com/programmers-and-chefs/ Ignoring warnings because they haven't *yet* manifested as a bug, or worse, because you *assume* that they haven't manifested as a bug, is about as sensible as ignoring the oil warning light on your car because the engine hasn't yet seized up. Regardless of which language the software is written in. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From p.f.moore at gmail.com Wed Aug 1 14:00:27 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 1 Aug 2018 19:00:27 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On Wed, 1 Aug 2018 at 18:43, Steven D'Aprano wrote: > > On Wed, 01 Aug 2018 16:22:16 +0100, Paul Moore wrote: > > > If they've reported to you that your code produces warnings under -b, > > your response can quite reasonably be "thanks for the information, we've > > reviewed our bytes/string handling and can confirm that it's safe, so > > there's no fixes needed in reportlab". > > I'm sorry, I don't understand this reasoning. (Perhaps I have missed > something.) Robin says his code runs under both Python2 and Python3. He's > getting a warning that the behaviour has changed between the two, and > there's a dubious comparison being made between bytes and strings. > Consequently, there's a very real chance that he has a dicts which have > one key in Python 2 but two in Python 3: Rightly or wrongly, I'm trusting Robin's assertion that he doesn't believe there's a problem with his code. After all, the change in behaviour between Python 2 and 3 has been explained a couple of times in this thread, so I'm assuming Robin understands it, and is not simply asserting unverified that he thinks his code is OK. Certainly, if Robin hasn't checked that the warning isn't flagging an actual issue with his code, then he should do so. Is that not obvious? If it's not, then my apologies for assuming it was. My point was that it's a *warning*, and as such it's perfectly possible for a warning to *not* need addressing (other than to suppress or ignore it once you're happy that doing so is the right approach). Paul From rosuav at gmail.com Wed Aug 1 14:03:21 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 2 Aug 2018 04:03:21 +1000 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On Thu, Aug 2, 2018 at 3:37 AM, Steven D'Aprano wrote: > In another post, Chris says: > > I suspect that there may be a bit of non-thinking-C-mentality > creeping in: "if I can turn on warnings, I should, and any > warning is a problem". That simply isn't the case in Python. > > I strongly disagree. Unless Chris' intention is to say bugs don't matter > if they're written in Python, I don't know why one would say that > warnings aren't a problem. > > Every warning is one of three cases: > > - it reveals an existing actual problem; > > - it reveals a potential problem which might somebody become > an actual problem; > > - or it is a false positive which (if unfixed) distracts > attention and encourages a blas? attitude which could > easily lead to problems in the future. Right, exactly. And in any code that does not and cannot run on Python 2, the warning about bytes and text comparing unequal is nothing more than a false positive. It's like having a warning saying "In previous versions of Python, this arithmetic operation might have overflowed". In ALL current versions of Python, that is irrelevant noise. Turning on the warning is therefore not a good thing; the non-thinking policy of "turn on all warnings" is not the best default. If a warning is ALWAYS a false positive, it is better to silence it than to have noise that makes *other* warnings less visible. ChrisA From storchaka at gmail.com Wed Aug 1 15:14:54 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Wed, 1 Aug 2018 22:14:54 +0300 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: 01.08.18 21:03, Chris Angelico ????: > And in any code that does not and cannot run on Python > 2, the warning about bytes and text comparing unequal is nothing more > than a false positive. Not always. If your code supported Python 2 in the past, or third-party dependencies supports or supported Python 2, this warning can expose a real bug. Even if all your and third-party code always was Python 3 only, the standard library can contain such kind of bugs. Several years after the EOL of Python 2.7 and moving all living code to Python 3 we can ignore bytes warnings as always false positive. From cseberino at gmail.com Wed Aug 1 16:11:10 2018 From: cseberino at gmail.com (cseberino at gmail.com) Date: Wed, 1 Aug 2018 13:11:10 -0700 (PDT) Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process Message-ID: I can run python3 interactively in a subprocess w/ Popen but if I sent it text, that throws an exception, the process freezes instead of just printing the exception like the normal interpreter.. why? how fix? Here is my code below. (I suspect when there is an exception, there is NO output to stdin so that the problem is the line below that tries to read from stdin never finishes. Maybe I need a different readline that can "survive" when there is no output and won't block?) .... import subprocess interpreter = subprocess.Popen(['python3', '-i'], stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE) while True: exp = input(">>> ").encode() + b"\n" interpreter.stdin.write(exp) interpreter.stdin.flush() print(interpreter.stdout.readline().strip()) interpreter.stdin.close() interpreter.terminate() From rosuav at gmail.com Wed Aug 1 16:35:01 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 2 Aug 2018 06:35:01 +1000 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: On Thu, Aug 2, 2018 at 6:11 AM, wrote: > I can run python3 interactively in a subprocess w/ Popen but > if I sent it text, that throws an exception, the process freezes > instead of just printing the exception like the normal interpreter.. > why? how fix? Here is my code below. > > (I suspect when there is an exception, there is NO output to stdin so that > the problem is the line below that tries to read from stdin never finishes. > Maybe I need a different readline that can "survive" when there is no output and won't block?) > > .... > > import subprocess > > interpreter = subprocess.Popen(['python3', '-i'], > stdin = subprocess.PIPE, > stdout = subprocess.PIPE, > stderr = subprocess.PIPE) > > while True: > exp = input(">>> ").encode() + b"\n" > interpreter.stdin.write(exp) > interpreter.stdin.flush() > print(interpreter.stdout.readline().strip()) Normally, I would treat the input and output pipes separately. One easy (if potentially inefficient) way to do this is to create a thread for each of the pipes coming back - stdin and stderr, in this case - and have each one read from one pipe and display it. Then your main thread can ignore stdout and just push info into the stdin pipe. > interpreter.stdin.close() > interpreter.terminate() Side point: I would recommend joining, rather than forcibly terminating, the subprocess. Close stdin, then wait for it to finish; if you're paranoid, wait a limited amount of time and THEN terminate it, but otherwise, just join() the process. ChrisA From David.Raymond at tomtom.com Wed Aug 1 16:57:53 2018 From: David.Raymond at tomtom.com (David Raymond) Date: Wed, 1 Aug 2018 20:57:53 +0000 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: A couple notes: -I think the Python interpreter actually sends its output to stderr, so to capture it you'd probably want it to go to the same place as stdout, so use stderr = subprocess.STDOUT -You're only reading 1 line out output for each thing, so if 1 command creates multiple lines of output then you won't be showing them all. -You're never actually checking if the called process is still alive or not. It should normally be something like: ... interpreter = subprocess.Popen(...) ... interpreter.poll() while interpreter.returncode is not None: ... interpreter.poll() cleanup stuff -To the actual question it looks like it has the stdout stream in blocking mode somehow. So when you're reading from stdout and there's nothing there it's blocking and waiting for there to be something, which will never happen. Flipping through the documentation for subprocess, and io (interpreter.stdout is of ) io mentions blocking vs non blocking a lot, but it's not mentioned in subprocess. And I don't see in either how to tell if a stream is in blocking mode or not, or how or if it's possible to change that. So I don't know what to suggest for that, sorry. -----Original Message----- From: Python-list [mailto:python-list-bounces+david.raymond=tomtom.com at python.org] On Behalf Of cseberino at gmail.com Sent: Wednesday, August 01, 2018 4:11 PM To: python-list at python.org Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process I can run python3 interactively in a subprocess w/ Popen but if I sent it text, that throws an exception, the process freezes instead of just printing the exception like the normal interpreter.. why? how fix? Here is my code below. (I suspect when there is an exception, there is NO output to stdin so that the problem is the line below that tries to read from stdin never finishes. Maybe I need a different readline that can "survive" when there is no output and won't block?) .... import subprocess interpreter = subprocess.Popen(['python3', '-i'], stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE) while True: exp = input(">>> ").encode() + b"\n" interpreter.stdin.write(exp) interpreter.stdin.flush() print(interpreter.stdout.readline().strip()) interpreter.stdin.close() interpreter.terminate() -- https://mail.python.org/mailman/listinfo/python-list From flebber.crue at gmail.com Wed Aug 1 17:19:18 2018 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Wed, 1 Aug 2018 14:19:18 -0700 (PDT) Subject: Use a function arg in soup In-Reply-To: References: <223d4208-79b2-4e21-a6bd-37e5fda960ce@googlegroups.com> Message-ID: <4c1fa5cb-b8c4-440f-be87-9febe16a4725@googlegroups.com> Thanks Peter ### (2) attrs is a dict, so iterating over it will lose the values. Are you sure you want that? ### Yes initially I want just the keys as a list so I can choose to filter them out to only the ones I want. # getAttr Thanks very much will get my function up and working. Cheers Sayth From tjreedy at udel.edu Wed Aug 1 20:51:42 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 1 Aug 2018 20:51:42 -0400 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: On 8/1/2018 4:11 PM, cseberino at gmail.com wrote: > I can run python3 interactively in a subprocess w/ Popen but > if I sent it text, that throws an exception, the process freezes > instead of just printing the exception like the normal interpreter.. > why? how fix? Here is my code below. > > (I suspect when there is an exception, there is NO output to stdin so that > the problem is the line below that tries to read from stdin never finishes. > Maybe I need a different readline that can "survive" when there is no output and won't block?) > > .... > > import subprocess > > interpreter = subprocess.Popen(['python3', '-i'], > stdin = subprocess.PIPE, > stdout = subprocess.PIPE, > stderr = subprocess.PIPE) > > while True: > exp = input(">>> ").encode() + b"\n" > interpreter.stdin.write(exp) > interpreter.stdin.flush() > print(interpreter.stdout.readline().strip()) subprocess is not meant for interaction through the pipes. That is why, I have been told, IDLE uses a socket for interaction. Multiprocess is apparently better suited for interaction without resorting to a socket. > interpreter.stdin.close() > interpreter.terminate() > -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Thu Aug 2 03:16:49 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 2 Aug 2018 07:16:49 +0000 (UTC) Subject: Are dicts supposed to raise comparison errors References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On Wed, 01 Aug 2018 19:00:27 +0100, Paul Moore wrote: [...] > My point was that it's a *warning*, and as such it's perfectly possible > for a warning to *not* need addressing (other than to suppress or ignore > it once you're happy that doing so is the right approach). And my point was that ignoring warnings is not the right approach. Suppressing them on a case-by-case basis (if possible) is one thing, but a blanket suppression goes to far, for the reasons I gave earlier. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From steve+comp.lang.python at pearwood.info Thu Aug 2 03:20:38 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 2 Aug 2018 07:20:38 +0000 (UTC) Subject: Are dicts supposed to raise comparison errors References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On Wed, 01 Aug 2018 22:14:54 +0300, Serhiy Storchaka wrote: > 01.08.18 21:03, Chris Angelico ????: >> And in any code that does not and cannot run on Python 2, the warning >> about bytes and text comparing unequal is nothing more than a false >> positive. > > Not always. If your code supported Python 2 in the past, or third-party > dependencies supports or supported Python 2, this warning can expose a > real bug. Even if all your and third-party code always was Python 3 > only, the standard library can contain such kind of bugs. > > Several years after the EOL of Python 2.7 and moving all living code to > Python 3 we can ignore bytes warnings as always false positive. Even then, I don't know that we should do that. I do not believe that the EOL of Python 2 will end all confusion between byte strings and text strings. There is ample opportunity for code to accidentally compare bytes and text even in pure Python 3 code, e.g. comparing data read from files reading from files which are supposed to be opened in the same binary/text mode but aren't. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From leo at superlel.me Thu Aug 2 03:46:04 2018 From: leo at superlel.me (=?UTF-8?Q?L=c3=a9o_El_Amri?=) Date: Thu, 2 Aug 2018 09:46:04 +0200 Subject: asyncio: Warning message when waiting for an Event set by AbstractLoop.add_reader Message-ID: <09d9c579-7d03-f740-1b40-48535afdeafa@superlel.me> Hello list, During my attempt to bring asyncio support to the multiprocessing Queue, I found warning messages when executing my code with asyncio debug logging enabled. It seems that awaiting for an Event will make theses messages appears after the second attempt to wait for the Event. Here is a code sample to test this (47 lines long): import asyncio import threading import os import io import time import logging logging.basicConfig() logging.getLogger('asyncio').setLevel(logging.DEBUG) fd1, fd2 = os.pipe() # (r, w) loop = asyncio.get_event_loop() event = asyncio.Event() def readfd(fd, size): buf = io.BytesIO() handle = fd remaining = size while remaining > 0: chunk = os.read(handle, remaining) n = len(chunk) if n == 0: raise Exception() buf.write(chunk) remaining -= n return buf.getvalue() def f(): loop.add_reader(fd1, event.set) loop.run_until_complete(g()) loop.close() async def g(): while True: await event.wait() msg = readfd(fd1, 4) event.clear() if msg == b'STOP': break print(msg) if __name__ == '__main__': t = threading.Thread(target=f) t.start() time.sleep(1) os.write(fd2, b'TST1') time.sleep(1) os.write(fd2, b'TST2') time.sleep(1) os.write(fd2, b'TST3') time.sleep(1) os.write(fd2, b'STOP') t.join() I get this output: DEBUG:asyncio:Using selector: EpollSelector DEBUG:asyncio:poll took 999.868 ms: 1 events b'TST1' b'TST2' WARNING:asyncio:Executing took 1.001 seconds INFO:asyncio:poll took 1000.411 ms: 1 events b'TST3' WARNING:asyncio:Executing took 1.001 seconds DEBUG:asyncio:Close <_UnixSelectorEventLoop [CUT]> I don't get the message, because it seems that the event is awaited properly. I'm especially bugged, because I get this output from the actual code I'm writing: WARNING:asyncio:Executing took 1.000 seconds DEBUG:asyncio:poll took 0.012 ms: 1 events Here, it seems that "polling" went for 0.012 ms. But it should have gone for 1000.0 ms. But in any case, it still works. However, the fact that polling seems to "return" instantly is a strange fact. The test code is not producing this difference. In fact, it seems that polling is working as intended (Polling takes 1000.0 ms and Event is awaited for 1000.0 ms). But there is still this warning... Maybe one of you have an insight on this ? From p.f.moore at gmail.com Thu Aug 2 04:46:03 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 2 Aug 2018 09:46:03 +0100 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: On Wed, 1 Aug 2018 at 21:17, wrote: > > I can run python3 interactively in a subprocess w/ Popen but > if I sent it text, that throws an exception, the process freezes > instead of just printing the exception like the normal interpreter.. > why? how fix? Here is my code below. > > (I suspect when there is an exception, there is NO output to stdin so that > the problem is the line below that tries to read from stdin never finishes. > Maybe I need a different readline that can "survive" when there is no output and won't block?) > > .... > > import subprocess > > interpreter = subprocess.Popen(['python3', '-i'], > stdin = subprocess.PIPE, > stdout = subprocess.PIPE, > stderr = subprocess.PIPE) > > while True: > exp = input(">>> ").encode() + b"\n" > interpreter.stdin.write(exp) > interpreter.stdin.flush() > print(interpreter.stdout.readline().strip()) > interpreter.stdin.close() > interpreter.terminate() You're only reading one line from stdout, but an exception is multiple lines. So the subprocess is still trying to write while you're wanting to give it input again. This is a classic way to get a deadlock, which is basically what you're seeing. Add to that the fact that there are likely IO buffers in the subprocess that mean it's not necessarily passing output back to you at the exact time you expect it to (and the subprocess probably has different buffering behaviour when the IO is to pipes rather than to the console) and it gets complex fast. As others have mentioned, separate threads for the individual pipes may help, or if you need to go that far there are specialised libraries, I believe (pexpect is one, but from what I know it's fairly Unix-specific, so I'm not very familiar with it). Sorry, but there's no "simple" answer here for you (although you may well be able to get something that works well enough for your specific needs - but it's not obvious from your snippet of code what you're trying to achieve). Paul From robin at reportlab.com Thu Aug 2 05:31:22 2018 From: robin at reportlab.com (Robin Becker) Date: Thu, 2 Aug 2018 10:31:22 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On 01/08/2018 18:19, Peter Otten wrote: ........ > > I've looked into the actual code which has > > # paraparser.py > f = isPy3 and asBytes or asUnicode > K = list(known_entities.keys()) > for k in K: > known_entities[f(k)] = known_entities[k] > > It looks like known_entities starts out with the default string type, i. e. > unicode in py3 and bytes in py2. > > While in py2 the code has no effect in py3 it adds the corresponding keys of > type bytes. However, known_entities is then used in > HTMLParser.handle_entity_ref(self, name) which passes the name as unicode in > py3. > > I didn't try, but I would suspect that the module keeps working as expected > when you remove the lines quoted above. > I did try and all the tests pass in 2.7.x & >=3.3; the commit message says "fix entityref handling" and happened during the python 3.x porting. I suppose there was some issue, but its entirely probable that some later change (eg parser) has fixed the original problem and made this code redundant. > If I'm correct running the program with the -b flag has at least helped in > cleaning up the code in this case. > and I will try running all tests under that flag; it's sure to find more issues. > In other cases it might detect sources of bugs, so IMHO it's better to have > a close look at and possibly rewrite code that triggers the warning rather > than to disable it. > -- Robin Becker From robin at reportlab.com Thu Aug 2 05:36:55 2018 From: robin at reportlab.com (Robin Becker) Date: Thu, 2 Aug 2018 10:36:55 +0100 Subject: Are dicts supposed to raise comparison errors In-Reply-To: References: <0f420694-b2e1-fb7f-319a-8399bc8af7dd@chamonix.reportlab.co.uk> <6cddaf24-2fdd-0b8d-142a-15a5b7368edc@mrabarnett.plus.com> <42becc8b-db24-58b6-c9f3-11108a6d1eeb@Damon-Family.org> <2d3a6abc-f743-4f4a-a6c8-f089219c5a3e@chamonix.reportlab.co.uk> Message-ID: On 02/08/2018 08:20, Steven D'Aprano wrote: > On Wed, 01 Aug 2018 22:14:54 +0300, Serhiy Storchaka wrote: > ........... >> Not always. If your code supported Python 2 in the past, or third-party >> dependencies supports or supported Python 2, this warning can expose a >> real bug. Even if all your and third-party code always was Python 3 >> only, the standard library can contain such kind of bugs. >> >> Several years after the EOL of Python 2.7 and moving all living code to >> Python 3 we can ignore bytes warnings as always false positive. > > Even then, I don't know that we should do that. I do not believe that the > EOL of Python 2 will end all confusion between byte strings and text > strings. There is ample opportunity for code to accidentally compare > bytes and text even in pure Python 3 code, e.g. comparing data read from > files reading from files which are supposed to be opened in the same > binary/text mode but aren't. > > I think I agree; when python 2 is history I can drop all the messing about with bytes in the input and clean up the code a lot. Stuff like svg and xml will still need conversions which are generally unknown in advance. The output will need converting to bytes, but that's fairly standard. -- Robin Becker From rosuav at gmail.com Thu Aug 2 05:42:40 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 2 Aug 2018 19:42:40 +1000 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: On Thu, Aug 2, 2018 at 6:46 PM, Paul Moore wrote: > On Wed, 1 Aug 2018 at 21:17, wrote: >> >> I can run python3 interactively in a subprocess w/ Popen but >> if I sent it text, that throws an exception, the process freezes >> instead of just printing the exception like the normal interpreter.. >> why? how fix? Here is my code below. >> >> (I suspect when there is an exception, there is NO output to stdin so that >> the problem is the line below that tries to read from stdin never finishes. >> Maybe I need a different readline that can "survive" when there is no output and won't block?) >> >> .... >> >> import subprocess >> >> interpreter = subprocess.Popen(['python3', '-i'], >> stdin = subprocess.PIPE, >> stdout = subprocess.PIPE, >> stderr = subprocess.PIPE) >> >> while True: >> exp = input(">>> ").encode() + b"\n" >> interpreter.stdin.write(exp) >> interpreter.stdin.flush() >> print(interpreter.stdout.readline().strip()) >> interpreter.stdin.close() >> interpreter.terminate() > > You're only reading one line from stdout, but an exception is multiple > lines. So the subprocess is still trying to write while you're wanting > to give it input again. This is a classic way to get a deadlock, which > is basically what you're seeing. Add to that the fact that there are > likely IO buffers in the subprocess that mean it's not necessarily > passing output back to you at the exact time you expect it to (and the > subprocess probably has different buffering behaviour when the IO is > to pipes rather than to the console) and it gets complex fast. > > As others have mentioned, separate threads for the individual pipes > may help, or if you need to go that far there are specialised > libraries, I believe (pexpect is one, but from what I know it's fairly > Unix-specific, so I'm not very familiar with it). Another possibility: If the ONLY thing you're doing with stdout/stderr is passing them through to the screen, simply don't change them. Let them remain bound to the console. You can have a pipe for stdin without also having pipes for the others. But that won't work if you intend to do any sort of parsing on the returned output. ChrisA From arj.python at gmail.com Thu Aug 2 09:24:29 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Thu, 2 Aug 2018 17:24:29 +0400 Subject: image in db to string to actual image Message-ID: storing images in db is easy but to retrieve them natively how to? (ignore db type, using orm). meaning without PIL / Pillow. type : png like i'd like to have it as string, any idea ? Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius From larry at hastings.org Thu Aug 2 10:00:11 2018 From: larry at hastings.org (Larry Hastings) Date: Thu, 2 Aug 2018 07:00:11 -0700 Subject: [RELEASED] Python 3.4.9 and Python 3.5.6 are now available Message-ID: On behalf of the Python development community, I'm happy to announce the availability of Python 3.4.9 and Python 3.5.6. Both Python 3.4 and 3.5 are in "security fixes only" mode.? Both versions only accept security fixes, not conventional bug fixes, and both releases are source-only. You can find Python 3.4.9 here: https://www.python.org/downloads/release/python-349/ And you can find Python 3.5.6 here: https://www.python.org/downloads/release/python-356/ We now return you to your pitched debate already in progress, //arry/ From wanderer at dialup4less.com Thu Aug 2 12:02:44 2018 From: wanderer at dialup4less.com (Wanderer) Date: Thu, 2 Aug 2018 09:02:44 -0700 (PDT) Subject: Problems with wxPython _core_.pyd on windows98 Message-ID: <89004d86-448d-4d27-ba47-468660edc96b@googlegroups.com> I have a laptop with windows 98 I use to connect to the OBD2 port on my car. I'm trying to install pyobd. I have a build for Python 2.7 for Windows98 that works but I'm having trouble with running wxPython. I get the following error. C:\pyobd>python pyobd.py Traceback (most recent call last): File "pyobd.py", line 28, in import wx File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\__init__.py", line 45, in from wx_core import * File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\_core.py", line4, in import _core_ Import Error: DLL load failed: A device attached to the system is not functioning. So what is going on? Do these pyd files need to be compiled specifically for win98? If so how do I do that? Does python compile these dlls or do I need a C compiler? Is there an old version of wxPython that will work on windows 98? Thanks From arj.python at gmail.com Thu Aug 2 12:35:01 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Thu, 2 Aug 2018 20:35:01 +0400 Subject: image in db to string to actual image In-Reply-To: References: Message-ID: ah those can be used, thanks, and to write the actual image? On Thu, Aug 2, 2018 at 8:20 PM Gilmeh Serda wrote: > On Thu, 02 Aug 2018 17:24:29 +0400, Abdur-Rahmaan Janhangeer wrote: > > > like i'd like to have it as string, any idea ? > > UUEncode / Base64 / yEnc ...? > > -- > Gilmeh > -- > https://mail.python.org/mailman/listinfo/python-list > -- Abdur-Rahmaan Janhangeer https://github.com/abdur-rahmaanj Mauritius From arj.python at gmail.com Thu Aug 2 14:35:10 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Thu, 2 Aug 2018 22:35:10 +0400 Subject: beware of linked in - mail used on this list Message-ID: just an info if you are using the mail you use in this list for linked in you might get surprises apologies if you got a mail from linkedin somewhere Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius From wanderer at dialup4less.com Thu Aug 2 15:46:04 2018 From: wanderer at dialup4less.com (Wanderer) Date: Thu, 2 Aug 2018 12:46:04 -0700 (PDT) Subject: Problems with wxPython _core_.pyd on windows98 In-Reply-To: <89004d86-448d-4d27-ba47-468660edc96b@googlegroups.com> References: <89004d86-448d-4d27-ba47-468660edc96b@googlegroups.com> Message-ID: <6facc390-f9cf-4ea4-964d-d5c94ee562bc@googlegroups.com> On Thursday, August 2, 2018 at 12:03:01 PM UTC-4, Wanderer wrote: > I have a laptop with windows 98 I use to connect to the OBD2 port on my car. I'm trying to install pyobd. I have a build for Python 2.7 for Windows98 that works but I'm having trouble with running wxPython. I get the following error. > > C:\pyobd>python pyobd.py > Traceback (most recent call last): > File "pyobd.py", line 28, in > import wx > File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\__init__.py", line 45, in > from wx_core import * > File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\_core.py", line4, in > import _core_ > Import Error: DLL load failed: A device attached to the system is not functioning. > > > So what is going on? > > Do these pyd files need to be compiled specifically for win98? If so how do I do that? Does python compile these dlls or do I need a C compiler? > > Is there an old version of wxPython that will work on windows 98? > > Thanks I got it working with Python 2.5.1 pywin32-207 pyserial 2.4 wxPython2.7 Though I'm still getting a warning API version mismatch for module win32event. This Python has API version 1013, module win32event has version 1012 I don't know what it means. pyobd.py has bug in it line 354 if "OS" in os.environ.keys() : running windows "OS" is not in os.environ.keys() in windows 98, so I changed "OS" to "WINDIR" which is in os environ.keys in windows 98. From cseberino at gmail.com Thu Aug 2 15:51:04 2018 From: cseberino at gmail.com (cseberino at gmail.com) Date: Thu, 2 Aug 2018 12:51:04 -0700 (PDT) Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: <1b170c30-d496-4127-8d17-67d41ffa6081@googlegroups.com> > -I think the Python interpreter actually sends its output to stderr, so to capture it you'd probably want it to go to the same place as stdout, so use stderr = subprocess.STDOUT Yes that captured the error messages! Thanks! > -You're only reading 1 line out output for each thing, so if 1 command creates multiple lines of output then you won't be showing them all. Yes. That is the one remaining problem. I tried replacing the readline with read and readlines and both froze/blocked. cs From cseberino at gmail.com Thu Aug 2 15:52:43 2018 From: cseberino at gmail.com (cseberino at gmail.com) Date: Thu, 2 Aug 2018 12:52:43 -0700 (PDT) Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: > subprocess is not meant for interaction through the pipes. That is why, > I have been told, IDLE uses a socket for interaction. Multiprocess is > apparently better suited for interaction without resorting to a socket. So use normal socket on localhost for this? Don't you still need subprocess to launch the interpreter in a separate process? cs From cseberino at gmail.com Thu Aug 2 15:54:39 2018 From: cseberino at gmail.com (cseberino at gmail.com) Date: Thu, 2 Aug 2018 12:54:39 -0700 (PDT) Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: > As others have mentioned, separate threads for the individual pipes > may help, or if you need to go that far there are specialised > libraries, I believe (pexpect is one, but from what I know it's fairly > Unix-specific, so I'm not very familiar with it). I'm on Linux so pexpect is a possibility. > Sorry, but there's no "simple" answer here for you (although you may > well be able to get something that works well enough for your specific > needs - but it's not obvious from your snippet of code what you're > trying to achieve). To send and receive text from a subprocess..even when there are exceptions. From cseberino at gmail.com Thu Aug 2 15:55:17 2018 From: cseberino at gmail.com (cseberino at gmail.com) Date: Thu, 2 Aug 2018 12:55:17 -0700 (PDT) Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: > Another possibility: If the ONLY thing you're doing with stdout/stderr > is passing them through to the screen, simply don't change them. Let > them remain bound to the console. You can have a pipe for stdin > without also having pipes for the others. But that won't work if you > intend to do any sort of parsing on the returned output. I must parse. From p.f.moore at gmail.com Thu Aug 2 17:45:33 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 2 Aug 2018 22:45:33 +0100 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: On Thu, 2 Aug 2018 at 20:58, wrote: > > Sorry, but there's no "simple" answer here for you (although you may > > well be able to get something that works well enough for your specific > > needs - but it's not obvious from your snippet of code what you're > > trying to achieve). > > To send and receive text from a subprocess..even when there are exceptions. For an arbitrary program as the subprocess? You need some sort of threading in your process, or maybe some sort of async processing with non-blocking reads/writes, as you cannot assume any particular interleaving of input and output on the part of the child. If you control the child process, you can implement a custom protocol to do the communication, and avoid many of the problems that way. If you're trying to "wrap" something like the Python interpreter, it's certainly possible, but it's tricky to get all the edge cases right (I know, I've tried!) and you probably need to run it with the -u flag (or with PYTHONUNBUFFERED set) to avoid IO buffers making your life miserable ;-) Or, as you say you're on Unix, you may be able to use ptys to do this - I gather that's what they are designed for, but as a Windows programmer myself, I know very little about them. Paul From steve+comp.lang.python at pearwood.info Thu Aug 2 20:53:16 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 3 Aug 2018 00:53:16 +0000 (UTC) Subject: beware of linked in - mail used on this list References: Message-ID: On Thu, 02 Aug 2018 22:35:10 +0400, Abdur-Rahmaan Janhangeer wrote: > just an info if you are using the mail you use in this list for linked > in you might get surprises > > apologies if you got a mail from linkedin somewhere LinkedIn is a spammer. I frequently get friend requests from people who I don't know from LinkedIn, and most of them don't even know they sent them. I got three from you yesterday. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From arj.python at gmail.com Thu Aug 2 21:02:31 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Fri, 3 Aug 2018 05:02:31 +0400 Subject: beware of linked in - mail used on this list In-Reply-To: References: Message-ID: was supposed to be one i'm aware of but 3 in 1 day ? Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius From tjreedy at udel.edu Thu Aug 2 23:56:16 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 2 Aug 2018 23:56:16 -0400 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: On 8/2/2018 3:52 PM, cseberino at gmail.com wrote: > >> subprocess is not meant for interaction through the pipes. That is why, >> I have been told, IDLE uses a socket for interaction. Multiprocess is >> apparently better suited for interaction without resorting to a socket. > > So use normal socket on localhost for this? Don't you still need subprocess > to launch the interpreter in a separate process? Yes. And you need to replace sys.stdxxx with file-like objects that redirect streams from and to the socket. And it is not 100% reliable, just 99.99??% reliableC -- Terry Jan Reedy From dieter at handshake.de Fri Aug 3 01:05:20 2018 From: dieter at handshake.de (dieter) Date: Fri, 03 Aug 2018 07:05:20 +0200 Subject: asyncio: Warning message when waiting for an Event set by AbstractLoop.add_reader References: <09d9c579-7d03-f740-1b40-48535afdeafa@superlel.me> Message-ID: <8736vwqbu7.fsf@handshake.de> L?o El Amri via Python-list writes: > ... > WARNING:asyncio:Executing took 1.000 seconds > ... > But there is still this warning... At your place, I would look at the code responsible for the warning. I assume that it is produced because the waiting time is rather high -- but this is just a guess. From arj.python at gmail.com Fri Aug 3 03:18:33 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Fri, 3 Aug 2018 11:18:33 +0400 Subject: image in db to string to actual image In-Reply-To: References: Message-ID: no not ascii art, just an application as described. thank you very much ! Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius On Fri, 3 Aug 2018, 07:45 Gilmeh Serda, wrote: > On Thu, 02 Aug 2018 20:35:01 +0400, Abdur-Rahmaan Janhangeer wrote: > > > ah those can be used, thanks, and to write the actual image? > > pseudo code: > > store_to_whatever(path2bin): > my_storage.store(uuencode(path2bin)) > > get_image_from_storage(path2uue): > return uudecode(my_storage.read(path2uue)) > > RTM on uue and its siblings. > > Nb with uue you have to make about 30-40% more room for storage, yEnc is > the smallest, only adds about 5-10% to the output. I believe Mime/Base64 > is somewhere in between. > > Just thought of something: you didn't mean to interpret the image as > ASCII art, did you? UUE and stuff, are not for that. > > In that case, look into the software Jave (Java based). I think it might > be open source. > > -- > Gilmeh > -- > https://mail.python.org/mailman/listinfo/python-list > From mausg at mail.com Fri Aug 3 03:49:40 2018 From: mausg at mail.com (mausg at mail.com) Date: 3 Aug 2018 07:49:40 GMT Subject: NLTK Message-ID: I like to analyse text. my method consisted of something like words=text.split(), which would split the text into space-seperated units. then I tried to use the Python NLTK library, which had alot of features I wanted, but using `word-tokenize' gives a different answer.- What gives?. -- Maus at ireland.com Will Rant For Food From tjol at tjol.eu Fri Aug 3 03:55:50 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 3 Aug 2018 09:55:50 +0200 Subject: Problems with wxPython _core_.pyd on windows98 In-Reply-To: <89004d86-448d-4d27-ba47-468660edc96b@googlegroups.com> References: <89004d86-448d-4d27-ba47-468660edc96b@googlegroups.com> Message-ID: <098a57a8-e11e-82e1-87af-7e068300635f@tjol.eu> On 02/08/18 18:02, Wanderer wrote: > I have a laptop with windows 98 I use to connect to the OBD2 port on my car. I'm trying to install pyobd. I have a build for Python 2.7 for Windows98 that works but I'm having trouble with running wxPython. I get the following error. I'm sure you have your reasons, but of course Windows 98 hasn't been supported, by Python, Microsoft, or anybody else, for many years. Are you sure it wouldn't be easier to use a newer laptop or install Linux? Support for Windows 98 was dropped in Python 2.6. > > C:\pyobd>python pyobd.py > Traceback (most recent call last): > File "pyobd.py", line 28, in > import wx > File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\__init__.py", line 45, in > from wx_core import * > File "C:\Python27\lib\site-packages\wx-3.0-msw\wx\_core.py", line4, in > import _core_ > Import Error: DLL load failed: A device attached to the system is not functioning. > > > So what is going on? > > Do these pyd files need to be compiled specifically for win98? If so how do I do that? Does python compile these dlls or do I need a C compiler? If you want to compile extension modules, you need to C compiler Python was compiled with. I expect this is Visual C++ 6.0 for the last Win9x releases. > > Is there an old version of wxPython that will work on windows 98? > From stephane at wirtel.be Fri Aug 3 09:48:45 2018 From: stephane at wirtel.be (Stephane Wirtel) Date: Fri, 3 Aug 2018 15:48:45 +0200 Subject: image in db to string to actual image In-Reply-To: References: Message-ID: <20180803134845.GA28803@xps> And what is your database? Maybe there is an API for the blob field On 08/02, Abdur-Rahmaan Janhangeer wrote: >storing images in db is easy but to retrieve them natively how to? (ignore >db type, using orm). meaning without PIL / Pillow. type : png > >like i'd like to have it as string, any idea ? > >Abdur-Rahmaan Janhangeer >https://github.com/Abdur-rahmaanJ >Mauritius >-- >https://mail.python.org/mailman/listinfo/python-list -- St?phane Wirtel - https://wirtel.be - @matrixise From oscar.j.benjamin at gmail.com Fri Aug 3 10:03:22 2018 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Fri, 3 Aug 2018 15:03:22 +0100 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: On 2 August 2018 at 20:54, wrote: > >> As others have mentioned, separate threads for the individual pipes >> may help, or if you need to go that far there are specialised >> libraries, I believe (pexpect is one, but from what I know it's fairly >> Unix-specific, so I'm not very familiar with it). > > I'm on Linux so pexpect is a possibility. > >> Sorry, but there's no "simple" answer here for you (although you may >> well be able to get something that works well enough for your specific >> needs - but it's not obvious from your snippet of code what you're >> trying to achieve). > > To send and receive text from a subprocess..even when there are exceptions. You can do this without threads on Linux using select (or similar): https://docs.python.org/3.7/library/select.html#select.select -- Oscar From rosuav at gmail.com Fri Aug 3 10:06:15 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 4 Aug 2018 00:06:15 +1000 Subject: Dealing with errors in interactive subprocess running python interpreter that freeze the process In-Reply-To: References: Message-ID: On Sat, Aug 4, 2018 at 12:03 AM, Oscar Benjamin wrote: > On 2 August 2018 at 20:54, wrote: >> >>> As others have mentioned, separate threads for the individual pipes >>> may help, or if you need to go that far there are specialised >>> libraries, I believe (pexpect is one, but from what I know it's fairly >>> Unix-specific, so I'm not very familiar with it). >> >> I'm on Linux so pexpect is a possibility. >> >>> Sorry, but there's no "simple" answer here for you (although you may >>> well be able to get something that works well enough for your specific >>> needs - but it's not obvious from your snippet of code what you're >>> trying to achieve). >> >> To send and receive text from a subprocess..even when there are exceptions. > > You can do this without threads on Linux using select (or similar): > https://docs.python.org/3.7/library/select.html#select.select > Yes, but threads are simpler unless you already know how to use select. ChrisA From leo at superlel.me Fri Aug 3 17:00:20 2018 From: leo at superlel.me (=?UTF-8?Q?L=c3=a9o_El_Amri?=) Date: Fri, 3 Aug 2018 23:00:20 +0200 Subject: asyncio: Warning message when waiting for an Event set by AbstractLoop.add_reader In-Reply-To: <8736vwqbu7.fsf@handshake.de> References: <09d9c579-7d03-f740-1b40-48535afdeafa@superlel.me> <8736vwqbu7.fsf@handshake.de> Message-ID: <0418dff2-8707-00be-00b6-2fdbafaa751c@superlel.me> Thanks for the answer, but the problem is that this is happening in the built-in Event of the asyncio package; which is actually a coroutine. I don't expect the built-in to have this kind of behavior. I guess I'll have to dig on the source code of the asyncio default loop to actually understand how all the thing is behaving in the shadows. Maybe I'm simply doing something wrong. But it would mean that the documentation is lacking some details, or maybe that I'm just really stupid on this one. On 03/08/2018 07:05, dieter wrote: > L?o El Amri via Python-list writes: >> ... >> WARNING:asyncio:Executing took 1.000 seconds >> ... >> But there is still this warning... > > At your place, I would look at the code responsible for the warning. > I assume that it is produced because the waiting time is rather > high -- but this is just a guess. > From vincent.vande.vyvre at telenet.be Fri Aug 3 17:43:59 2018 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Fri, 3 Aug 2018 23:43:59 +0200 Subject: Fishing from PyPI ? Message-ID: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> I've received a mail askink me to confirm my identity on PyPI. The begining of the message: ------------------------------------- Your account VinsS on PyPI does not have a verified primary email address. Please take 2 minutes to verify your email: * To ensure that every email we send is going to its intended recipient * To ensure email sent from PyPI isn?t marked as spam by email clients and service * *If you do not verify, you will not be able to upload to PyPI, or be added as a maintainer to projects. You will no longer receive notifications from PyPI.* To verify, visit your Account Settings page. ------------------------------------- The Account Settings is : https://pypi.us18.list-manage.com/track/click?u=b96f861be540deb98fd700265&id=1763135c31&e=8824682d0f Phishing ? yes, no ? Vincent From sharan.basappa at gmail.com Sun Aug 5 09:05:46 2018 From: sharan.basappa at gmail.com (Sharan Basappa) Date: Sun, 5 Aug 2018 06:05:46 -0700 (PDT) Subject: machine learning forums Message-ID: <92c495e5-02df-487f-a330-2b699b9e4a3e@googlegroups.com> I am quite new to Python. I am learning Python as I am interested in machine learning. The issue is, I have not found any ML forum where novices like me can get help. I have tried reddit and each of my posts have gone unanswered. Looks like reddit forum prefers either abstract topics on ML or very complex issues for discussions. I have tried stackoverflow also but there only programming issues are entertained not to mention people trying to outwit each other at the expense of users like me. I would like to know if there are any ML forums where input can be provided for new users like me. From cs at cskk.id.au Sun Aug 5 19:28:13 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Mon, 6 Aug 2018 09:28:13 +1000 Subject: why doesn't an mmap.mmap object have some kind of memoryview method? Message-ID: <20180805232813.GA90221@cskk.homeip.net> I'm tinkering with a module which scans video files. The MP4 parser has a `discard_data` parameter which tells certain parts of the parser to discard some of the scanned data, particularly the MDAT box parse because I don't normally want to blow the machine's RAM on a huge video stream - I'm normally parsing out the semantic info such as metadata. However, I would _like_ to keep it available in a frugal fashion if I can, and it seems to me that the mmap module is ideal: map the file into memory, use the file itself as the backing store, and return views of the mmap. But slicing a mmap object returns a bytes, which I _imagine_ to be a copy of the file data instead of a view into the mapping (because bytes are supposed to be immutable, and in theory some other actor might change the file even though I myself have it mapped read only). It seems obvious to me that a method returning a memoryview of the mapped file would be very handy here: no data copies at all, and not even any I/O unless the data are accessed. But I see no such method in the documentation. Does anyone have any insight here? Cheers, Cameron Simpson From cs at cskk.id.au Sun Aug 5 19:44:01 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Mon, 6 Aug 2018 09:44:01 +1000 Subject: why doesn't an mmap.mmap object have some kind of memoryview method? In-Reply-To: <20180805232813.GA90221@cskk.homeip.net> References: <20180805232813.GA90221@cskk.homeip.net> Message-ID: <20180805234401.GA36785@cskk.homeip.net> I should add that I'm using CPython 3 on a Mac, happy to receive CPython 3 UNIX-specific advice. - Cameron On 06Aug2018 09:28, Cameron Simpson wrote: >I'm tinkering with a module which scans video files. The MP4 parser >has a `discard_data` parameter which tells certain parts of the parser >to discard some of the scanned data, particularly the MDAT box parse >because I don't normally want to blow the machine's RAM on a huge >video stream - I'm normally parsing out the semantic info such as >metadata. > >However, I would _like_ to keep it available in a frugal fashion if I >can, and it seems to me that the mmap module is ideal: map the file >into memory, use the file itself as the backing store, and return >views of the mmap. > >But slicing a mmap object returns a bytes, which I _imagine_ to be a >copy of the file data instead of a view into the mapping (because >bytes are supposed to be immutable, and in theory some other actor >might change the file even though I myself have it mapped read only). > >It seems obvious to me that a method returning a memoryview of the >mapped file would be very handy here: no data copies at all, and not >even any I/O unless the data are accessed. But I see no such method in >the documentation. > >Does anyone have any insight here? > >Cheers, >Cameron Simpson From eryksun at gmail.com Sun Aug 5 19:55:37 2018 From: eryksun at gmail.com (eryk sun) Date: Sun, 5 Aug 2018 23:55:37 +0000 Subject: why doesn't an mmap.mmap object have some kind of memoryview method? In-Reply-To: <20180805232813.GA90221@cskk.homeip.net> References: <20180805232813.GA90221@cskk.homeip.net> Message-ID: On Sun, Aug 5, 2018 at 11:28 PM, Cameron Simpson wrote: > > It seems obvious to me that a method returning a memoryview of the mapped > file would be very handy here: no data copies at all, and not even any I/O > unless the data are accessed. But I see no such method in the documentation. mmap objects support the buffer protocol. Pass the mmap object to built-in memoryview(). From cs at cskk.id.au Sun Aug 5 19:59:00 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Mon, 6 Aug 2018 09:59:00 +1000 Subject: why doesn't an mmap.mmap object have some kind of memoryview method? In-Reply-To: References: Message-ID: <20180805235900.GA71065@cskk.homeip.net> On 05Aug2018 23:55, eryk sun wrote: >On Sun, Aug 5, 2018 at 11:28 PM, Cameron Simpson wrote: >> It seems obvious to me that a method returning a memoryview of the mapped >> file would be very handy here: no data copies at all, and not even any I/O >> unless the data are accessed. But I see no such method in the documentation. > >mmap objects support the buffer protocol. Pass the mmap object to >built-in memoryview(). As I've just discovered not 30 seconds before seeing your post :-) Thanks, this is what I'll be doing! Cheers, Cameron Simpson From steve+comp.lang.python at pearwood.info Sun Aug 5 19:59:35 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 5 Aug 2018 23:59:35 +0000 (UTC) Subject: machine learning forums References: <92c495e5-02df-487f-a330-2b699b9e4a3e@googlegroups.com> Message-ID: On Sun, 05 Aug 2018 06:05:46 -0700, Sharan Basappa wrote: > I am quite new to Python. I am learning Python as I am interested in > machine learning. The issue is, I have not found any ML forum where > novices like me can get help. I have tried reddit and each of my posts > have gone unanswered. Which subreddits have you posted to? > Looks like reddit forum prefers either abstract > topics on ML or very complex issues for discussions. > > I have tried stackoverflow also but there only programming issues are > entertained I believe Stackoverflow has a dedicated machine-learning site, "Cross Validated": https://meta.stackexchange.com/questions/130524/which-stack-exchange- website-for-machine-learning-and-computational-algorithms https://meta.stackexchange.com/questions/227757/where-to-ask-basic- questions-about-machine-learning -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From python at mrabarnett.plus.com Sun Aug 5 20:15:33 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 6 Aug 2018 01:15:33 +0100 Subject: why doesn't an mmap.mmap object have some kind of memoryview method? In-Reply-To: <20180805232813.GA90221@cskk.homeip.net> References: <20180805232813.GA90221@cskk.homeip.net> Message-ID: <2b6c153f-fa10-4fdc-7690-19a9b40f45be@mrabarnett.plus.com> On 2018-08-06 00:28, Cameron Simpson wrote: > I'm tinkering with a module which scans video files. The MP4 parser has a > `discard_data` parameter which tells certain parts of the parser to discard > some of the scanned data, particularly the MDAT box parse because I don't > normally want to blow the machine's RAM on a huge video stream - I'm normally > parsing out the semantic info such as metadata. > > However, I would _like_ to keep it available in a frugal fashion if I can, and > it seems to me that the mmap module is ideal: map the file into memory, use the > file itself as the backing store, and return views of the mmap. > > But slicing a mmap object returns a bytes, which I _imagine_ to be a copy of > the file data instead of a view into the mapping (because bytes are supposed to > be immutable, and in theory some other actor might change the file even though > I myself have it mapped read only). > > It seems obvious to me that a method returning a memoryview of the mapped file > would be very handy here: no data copies at all, and not even any I/O unless > the data are accessed. But I see no such method in the documentation. > > Does anyone have any insight here? > The docs show that although slicing a mmap returns bytes, you can assign to a slice of a mmap itself. You can also give an offset and length into the file when you create the mmap. From downtime at thisisnotreal.com Sun Aug 5 23:50:52 2018 From: downtime at thisisnotreal.com (downtime) Date: Mon, 6 Aug 2018 03:50:52 +0000 (UTC) Subject: machine learning forums References: <92c495e5-02df-487f-a330-2b699b9e4a3e@googlegroups.com> Message-ID: You might also try courses on Udemy and Udacity. I know Udemy is always having sales on courses for like $10.99. They have beginner/novice courses for all kinds of topics and in my experience, there are some pretty good ones. On 2018-08-05, Steven D'Aprano wrote: > On Sun, 05 Aug 2018 06:05:46 -0700, Sharan Basappa wrote: > >> I am quite new to Python. I am learning Python as I am interested in >> machine learning. The issue is, I have not found any ML forum where >> novices like me can get help. I have tried reddit and each of my posts >> have gone unanswered. > > Which subreddits have you posted to? > > >> Looks like reddit forum prefers either abstract >> topics on ML or very complex issues for discussions. >> >> I have tried stackoverflow also but there only programming issues are >> entertained > > I believe Stackoverflow has a dedicated machine-learning site, "Cross > Validated": > > https://meta.stackexchange.com/questions/130524/which-stack-exchange- > website-for-machine-learning-and-computational-algorithms > > https://meta.stackexchange.com/questions/227757/where-to-ask-basic- > questions-about-machine-learning > > From sharan.basappa at gmail.com Mon Aug 6 12:02:24 2018 From: sharan.basappa at gmail.com (Sharan Basappa) Date: Mon, 6 Aug 2018 09:02:24 -0700 (PDT) Subject: machine learning forums In-Reply-To: References: <92c495e5-02df-487f-a330-2b699b9e4a3e@googlegroups.com> Message-ID: > > I am quite new to Python. I am learning Python as I am interested in > > machine learning. The issue is, I have not found any ML forum where > > novices like me can get help. I have tried reddit and each of my posts > > have gone unanswered. > > Which subreddits have you posted to? its called machine learning > > Looks like reddit forum prefers either abstract > > topics on ML or very complex issues for discussions. > > > > I have tried stackoverflow also but there only programming issues are > > entertained > > I believe Stackoverflow has a dedicated machine-learning site, "Cross > Validated": > > https://meta.stackexchange.com/questions/130524/which-stack-exchange- > website-for-machine-learning-and-computational-algorithms > > https://meta.stackexchange.com/questions/227757/where-to-ask-basic- > questions-about-machine-learning Thanks. This looks like a very good sight. I hope folks are friendlier than stackoverflow. From sharan.basappa at gmail.com Mon Aug 6 12:05:50 2018 From: sharan.basappa at gmail.com (Sharan Basappa) Date: Mon, 6 Aug 2018 09:05:50 -0700 (PDT) Subject: machine learning forums In-Reply-To: References: <92c495e5-02df-487f-a330-2b699b9e4a3e@googlegroups.com> Message-ID: On Monday, 6 August 2018 09:21:03 UTC+5:30, downtime wrote: > You might also try courses on Udemy and Udacity. I know Udemy is always > having sales on courses for like $10.99. They have beginner/novice > courses for all kinds of topics and in my experience, there are some > pretty good ones. Actually, I have already completed machine learning course by Andrew NG. And then I spent close to 3 months learning Python as I quickly realized than without Python, it is nearly impossible to do any coding in ML area. But the issue is, from time to time, I do get doubts. For Python, I have this forum but for ML, not much options. Now I know about cross validated though. From arj.python at gmail.com Mon Aug 6 12:38:09 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 6 Aug 2018 20:38:09 +0400 Subject: Fishing from PyPI ? In-Reply-To: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> References: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> Message-ID: me too received it but what makes you suspect? Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius On Mon, 6 Aug 2018, 20:35 Vincent Vande Vyvre, < vincent.vande.vyvre at telenet.be> wrote: > I've received a mail askink me to confirm my identity on PyPI. > > The begining of the message: > > ------------------------------------- > > Your account VinsS on PyPI > < > https://pypi.us18.list-manage.com/track/click?u=b96f861be540deb98fd700265&id=9a19d150cf&e=8824682d0f> > > does not have a verified primary email address. > > Please take 2 minutes to verify your email: > > * To ensure that every email we send is going to its intended recipient > * To ensure email sent from PyPI isn?t marked as spam by email clients > and service > * *If you do not verify, you will not be able to upload to PyPI, or be > added as a maintainer to projects. You will no longer receive > notifications from PyPI.* > > To verify, visit your Account Settings > < > https://pypi.us18.list-manage.com/track/click?u=b96f861be540deb98fd700265&id=1763135c31&e=8824682d0f > > > page. > > ------------------------------------- > > The Account Settings > < > https://pypi.us18.list-manage.com/track/click?u=b96f861be540deb98fd700265&id=1763135c31&e=8824682d0f> > > is : > > https://pypi.us18.list-manage.com/track/click?u=b96f861be540deb98fd700265&id=1763135c31&e=8824682d0f > > Phishing ? yes, no ? > > > Vincent > > -- > https://mail.python.org/mailman/listinfo/python-list > From python at mrabarnett.plus.com Mon Aug 6 13:29:23 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 6 Aug 2018 18:29:23 +0100 Subject: Fishing from PyPI ? In-Reply-To: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> References: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> Message-ID: <04fd213e-cfb4-9455-56d1-24dc6f86a352@mrabarnett.plus.com> On 2018-08-03 22:43, Vincent Vande Vyvre wrote: > I've received a mail askink me to confirm my identity on PyPI. > > The begining of the message: > > ------------------------------------- > > Your account VinsS on PyPI > > does not have a verified primary email address. > > Please take 2 minutes to verify your email: > > * To ensure that every email we send is going to its intended recipient > * To ensure email sent from PyPI isn?t marked as spam by email clients > and service > * *If you do not verify, you will not be able to upload to PyPI, or be > added as a maintainer to projects. You will no longer receive > notifications from PyPI.* > > To verify, visit your Account Settings > > page. > > ------------------------------------- > > The Account Settings > > is : > https://pypi.us18.list-manage.com/track/click?u=b96f861be540deb98fd700265&id=1763135c31&e=8824682d0f > > Phishing ? yes, no ? > If you want to be sure, ignore the links in the email, and check PyPI at the URL that you know is correct. Given that the email says "https://pypi.us18.list-manage.com" and PyPI is at "https://pypi.org/", it does look suspicious. From kwpolska at gmail.com Mon Aug 6 14:26:58 2018 From: kwpolska at gmail.com (Chris Warrick) Date: Mon, 6 Aug 2018 20:26:58 +0200 Subject: Fishing from PyPI ? In-Reply-To: <04fd213e-cfb4-9455-56d1-24dc6f86a352@mrabarnett.plus.com> References: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> <04fd213e-cfb4-9455-56d1-24dc6f86a352@mrabarnett.plus.com> Message-ID: On Mon, 6 Aug 2018 at 19:31, MRAB wrote: > > https://pypi.us18.list-manage.com/track/[snip] > If you want to be sure, ignore the links in the email, and check PyPI at > the URL that you know is correct. > > Given that the email says "https://pypi.us18.list-manage.com" and PyPI > is at "https://pypi.org/", it does look suspicious. > -- > https://mail.python.org/mailman/listinfo/python-list Those e-mails are legitimate. See [1] and [2]. The unusual domain is a common staple of Mailchimp, which is an e-mail newsletter platform (it was used to mail out the announcement), and they replace all links with tracking ones in their list-manage.com domain. (They also implement the GDPR in an anti-user/pro-spam fashion, but I digress.) [1]: https://status.python.org/incidents/nk7cyn2vh4wr [2]: https://github.com/pypa/warehouse/issues/3632 -- Chris Warrick PGP: 5EAAEA16 From rhodri at kynesim.co.uk Mon Aug 6 15:18:07 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 6 Aug 2018 20:18:07 +0100 Subject: NLTK In-Reply-To: References: Message-ID: <69f37e73-971b-ca10-2f1a-fbd92cc50644@kynesim.co.uk> On 03/08/18 08:49, mausg at mail.com wrote: > I like to analyse text. my method consisted of something like > words=text.split(), which would split the text into space-seperated > units. then I tried to use the Python NLTK library, which had alot > of features I wanted, but using `word-tokenize' gives a different > answer.- > > What gives?. I don't know or use NLTK and you haven't provided any examples, so I can't say for certain. However vague memories from the natural language work I did decades ago suggest that perhaps splitting text by spaces is a little too simplistic, and NLTK may be trying to do something more comprehensive. -- Rhodri James *-* Kynesim Ltd From greg.ewing at canterbury.ac.nz Mon Aug 6 18:49:24 2018 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 07 Aug 2018 10:49:24 +1200 Subject: Fishing from PyPI ? In-Reply-To: References: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> <04fd213e-cfb4-9455-56d1-24dc6f86a352@mrabarnett.plus.com> Message-ID: Chris Warrick wrote: > The unusual domain is a common staple of Mailchimp, which is an e-mail > newsletter platform (it was used to mail out the announcement), and > they replace all links with tracking ones in their list-manage.com > domain. Sounds like you need to find a mail service that doesn't screw around with the contents of your messages. This is really quite obnoxious, IMO. -- Greg From steve+comp.lang.python at pearwood.info Mon Aug 6 20:41:21 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 7 Aug 2018 00:41:21 +0000 (UTC) Subject: NLTK References: Message-ID: On Fri, 03 Aug 2018 07:49:40 +0000, mausg wrote: > I like to analyse text. my method consisted of something like > words=text.split(), which would split the text into space-seperated > units. In natural language, words are more complicated than just space-separated units. Some languages don't use spaces as a word delimiter. Some don't use word delimiters at all. Even in English, the we have *compound words* which exist in three forms: - open: "ice cream" - closed: "notebook" - hyphenated: "long-term" Recognising open compound words is difficult. "Real estate" is an open compound word, but "real cheese" and "my estate" are both two words. Another problem for English speakers is deciding whether to treat contractions as a single word, or split them? "don't" --> "do" "n't" "they'll" --> "they" "'ll" Punctuation marks should either be stripped out of sentences before splitting into words, or treated as distinct tokens. We don't want "tokens" and "tokens." to be treated as distinct words, just because one happened to fall at the end of a sentence and one didn't. > then I tried to use the Python NLTK library, which had alot of > features I wanted, but using `word-tokenize' gives a different > answer.- > > What gives?. I'm pretty sure the function isn't called "word-tokenize". That would mean "word subtract tokenize" in Python code. Do you mean word_tokenize? Have you compared the output of the two and looked at how they differ? If there is too much output to compare by eye, you could convert to sets and check the set difference. Or try reading the documentation for word_tokenize: http://www.nltk.org/api/nltk.tokenize.html#nltk.tokenize.treebank.TreebankWordTokenizer -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From ben+python at benfinney.id.au Tue Aug 7 02:01:26 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 07 Aug 2018 16:01:26 +1000 Subject: Good reason not to obfuscate URLs (was: Fishing from PyPI ?) References: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> Message-ID: <86tvo6oguh.fsf@benfinney.id.au> Vincent Vande Vyvre writes: > To verify, visit your Account Settings > > page. > > ------------------------------------- > > The Account Settings > > is : > https://pypi.us18.list-manage.com/track/click?u=[? personally-identifying information ?] > > Phishing ? yes, no ? It's impossible to tell, from those links alone. The links are obfuscated deliberately. What we can say for certain, is that following those links allows parties unknown, to track the fact you've followed that link, before you ever get to PyPI. You are right to be concerned. This is one good reason why I argue that link obfuscation like this is bad practice: we can't tell what domain they will redirect to, so there's no way to know before visiting the link whether it will go to a ?python.org? URL. Instead, sending people links that you want them to follow should be direct links. That way we can see where it is the person wants us to visit. As a bonus, we avoid more layers of surveillance that these man-inthe-middle providers like ?list-manage.com? try to gather about our online behaviour. -- \ ?Programs must be written for people to read, and only | `\ incidentally for machines to execute.? ?Abelson & Sussman, | _o__) _Structure and Interpretation of Computer Programs_ | Ben Finney From steve+comp.lang.python at pearwood.info Tue Aug 7 03:58:25 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 7 Aug 2018 07:58:25 +0000 (UTC) Subject: RFC -- custom operators Message-ID: Request for comments -- proposal to allow custom binary operators. I'll looking for comments on custom binary operators: would it be useful, if so, what use-cases do you have? The most obvious and often-requested use-case would be for a form of logical operator (AND, OR, XOR) that is distinct from the bitwise operators & | ^ but unlike the standard `and` and `or` operators, calls dunder methods. The proposal is to add custom operators. A placeholder syntax might be: spam OP eggs which would then delegate to special dunder methods __OP__ or __ROP__ similar to existing operators such as + and similar. I don't want to get into arguments about syntax, or implementation details, unless there is some interest in the functionality. Please focus on *functional requirements* only. (1) This proposal requires operators to be legal identifiers, such as "XOR" or "spam", not punctuation like % and absolutely not Unicode symbols like ? (2) For the sake of the functional requirements, assume that we can parse `spam OP eggs` without any ambiguity; (3) This only proposes binary infix operators, not unary prefix or postfix operators; infix: argument1 OP argument2 prefix: OP argument postfix: argument OP (4) This does not propose to allow the precedence to be set on a case-by-case basis. All custom operators will have the same precedence. (5) What should that precedence be? (6) This does not propose to set the associativity on a case-by-case basis. All custom operators will have the same associativity. (7) Should the operators be left-associative (like multiplication), right-associative (like exponentiation), or non-associative? # Left-associative: a OP b OP c # like (a OP b) op c # Right-associative: a OP b OP c # like a OP (b op c) In the last case, that would make chained custom operators intentionally ambiguous (and hence a SyntaxError) unless disambiguated with parentheses: # Non-associative: a OP b OP c # SyntaxError (a OP b) OP c # okay a OP (b OP c) # okay (8) This does not propose to support short-circuiting operators. I'm not interested in hearing theoretical arguments that every infix operator can be written as a function or method call. I know that. I'm interested in hearing about use-cases where the code is improved and made more expressive by using operator syntax and existing operators aren't sufficient. (If there aren't any such use-cases, then there's no need for custom operators.) Thoughts? -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From kwpolska at gmail.com Tue Aug 7 04:06:23 2018 From: kwpolska at gmail.com (Chris Warrick) Date: Tue, 7 Aug 2018 10:06:23 +0200 Subject: Fishing from PyPI ? In-Reply-To: References: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> <04fd213e-cfb4-9455-56d1-24dc6f86a352@mrabarnett.plus.com> Message-ID: On Tue, 7 Aug 2018 at 00:52, Gregory Ewing wrote: > > Chris Warrick wrote: > > The unusual domain is a common staple of Mailchimp, which is an e-mail > > newsletter platform (it was used to mail out the announcement), and > > they replace all links with tracking ones in their list-manage.com > > domain. > > Sounds like you need to find a mail service that doesn't > screw around with the contents of your messages. This is > really quite obnoxious, IMO. For the record, I?m not in any way affiliated with the PyPA. I?m just linking to official resources that prove the legitimacy of these e-mails. As for finding a better host, it?s not easy. MailChimp, as annoying as they might be, has built up a good reputation with mail hosts* (of course, there are a bunch of other services that have such reputation as well.) However, if you send the e-mail yourself, and big mail hosts notice you have sent a ton of e-mail, they will probably consider you a spammer and make your life harder. Especially if your e-mail server is misconfigured in even the slightest way. * https://mailchimp.com/features/email-delivery/ -- Chris Warrick PGP: 5EAAEA16 From marko at pacujo.net Tue Aug 7 04:17:20 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 07 Aug 2018 11:17:20 +0300 Subject: RFC -- custom operators References: Message-ID: <87y3ditwtr.fsf@elektro.pacujo.net> Steven D'Aprano : > (1) This proposal requires operators to be legal identifiers, > such as "XOR" or "spam", not punctuation like % and > absolutely not Unicode symbols like ? Oh, that's a let-down. Operator symbols get their expressive value from visual conciseness: life?{?1 ??.?3 4=+/,?1 0 1?.??1 0 1?.???} > (If there aren't any such use-cases, then there's no need for custom > operators.) > > Thoughts? I have never felt the need for custom operators in Python code. I believe introducing them will make it harder, not easier, to read code. Marko From p.f.moore at gmail.com Tue Aug 7 05:03:28 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 7 Aug 2018 10:03:28 +0100 Subject: RFC -- custom operators In-Reply-To: References: Message-ID: On Tue, 7 Aug 2018 at 09:03, Steven D'Aprano wrote: > I'll looking for comments on custom binary operators: would it be useful, > if so, what use-cases do you have? I've never found a need for custom binary operators. I can imagine some *theoretical* cases where they might be useful (but no actual use cases!) but those would almost certainly require relaxing one or more of the restrictions you listed, so they do not even count as theoretical support for your suggested proposal. Paul From rhodri at kynesim.co.uk Tue Aug 7 07:17:08 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 7 Aug 2018 12:17:08 +0100 Subject: RFC -- custom operators In-Reply-To: References: Message-ID: <00890664-286f-2089-475f-59fd878beac8@kynesim.co.uk> On 07/08/18 08:58, Steven D'Aprano wrote: > Request for comments -- proposal to allow custom binary operators. [snip] > (1) This proposal requires operators to be legal identifiers, > such as "XOR" or "spam", not punctuation like % and > absolutely not Unicode symbols like ? Probably wise. I'd personally be inclined to use expressive symbols because I, of course, am an entirely reasonable person whose every choice will be completely intuitive to everyone else (ho ho), but I can imagine other people succumbing to the temptation to trying turning Python in APL. Also names are probably easier to parse. > (2) For the sake of the functional requirements, assume that > we can parse `spam OP eggs` without any ambiguity; Seems like a fair assumption. > (3) This only proposes binary infix operators, not unary > prefix or postfix operators; > > infix: argument1 OP argument2 > prefix: OP argument > postfix: argument OP Other than being easier to parse, is there any particular reason for this choice? > (4) This does not propose to allow the precedence to be > set on a case-by-case basis. All custom operators will > have the same precedence. > > (5) What should that precedence be? Low. Names, with their mandatory surrounding spaces, feel more separated and hence lower priority than their operands. I would expect x + 1 FROBNICATE q * 4 - 2 to parse as (x + 1) FROBNICATE (q*4-2) > (6) This does not propose to set the associativity on a > case-by-case basis. All custom operators will have > the same associativity. > > (7) Should the operators be left-associative (like multiplication), > right-associative (like exponentiation), or non-associative? Either left-associative or non-associative, probably the latter. Most operators that I can think of are left-associative, but you probably want non-associative to avoid subtle bugs with more naturally right-associative operators. > (If there aren't any such use-cases, then there's no need for custom > operators.) I can't think of any use cases I actually want off-hand. The sorts of things I might apply new operators to are various bits of byte buffer mangling, and most of those feel more pythonic as iterative processes. -- Rhodri James *-* Kynesim Ltd From giannis.dafnomilis at gmail.com Tue Aug 7 11:33:20 2018 From: giannis.dafnomilis at gmail.com (giannis.dafnomilis at gmail.com) Date: Tue, 7 Aug 2018 08:33:20 -0700 (PDT) Subject: How do I create a variable where one index depends on the value of another index? Message-ID: Hello guys. I'm having an issue with a Python PulP MILP problem. You can find the simplified code that reproduces the problem here: from pulp import * machines = 2 I = range(machines) positions = 2 J = range(positions) years = 10 T = range(years) age = {0: 5, 1: 7} IR = 0.06 df = 0.3 costs = {(0,0):300, (0,1):200, (1,0):500, (1,1):350} factor = {} finalcosts = {} for i in I: for j in J: for t in T: for k in range(age[i]): factor[t,k] = ((1-df)**k)/((1+IR)**t) finalcosts[i,j,t,k] = costs[i,j]*factor[t,k] prob = LpProblem("TrialProb",LpMinimize) Prob_vars = LpVariable.dicts("probvars", ((Machine, Position,Year, Age) for Machine in I for Position in J for Year in T for Age in range(age[i])),0,None, LpInteger) This gives me a 'finalcosts' variable with a size of 240 which is what I want, with all the correct values. But the 'Prob_vars' are of a size 260, counting the second index k for the first index i as well. Meaning that while in 'finalcosts' for i=0, k=0:4 and for i=2, k=0:6 (which is what I want), for the 'Prob_vars' decision variable index k=0:6 for both i=1 & i=2. I'm fairly new to Python so I can't quite grasp where the problem lies. What I have tried: I've tried all combinations that I could think of for different expressions of the 'Prob_vars' but nothing works properly. I also looked anywhere I could think of online but I can't find an answer to this. From __peter__ at web.de Tue Aug 7 13:30:09 2018 From: __peter__ at web.de (Peter Otten) Date: Tue, 07 Aug 2018 19:30:09 +0200 Subject: How do I create a variable where one index depends on the value of another index? References: Message-ID: giannis.dafnomilis at gmail.com wrote: > Hello guys. I'm having an issue with a Python PulP MILP problem. You can > find the simplified code that reproduces the problem here: > > from pulp import * > machines = 2 > I = range(machines) > positions = 2 > J = range(positions) > years = 10 > T = range(years) > age = {0: 5, 1: 7} > > IR = 0.06 > df = 0.3 > > costs = {(0,0):300, (0,1):200, (1,0):500, (1,1):350} > > factor = {} > finalcosts = {} > for i in I: > for j in J: > for t in T: > for k in range(age[i]): > factor[t,k] = ((1-df)**k)/((1+IR)**t) > finalcosts[i,j,t,k] = costs[i,j]*factor[t,k] > > prob = LpProblem("TrialProb",LpMinimize) > > Prob_vars = LpVariable.dicts("probvars", ((Machine, Position,Year, Age) > for Machine in I for Position in J for Year in T for Age in > range(age[i])),0,None, LpInteger) That should probably be age[Machine], not age[i] in the line above. > This gives me a 'finalcosts' variable with a size of 240 which is what I > want, with all the correct values. But the 'Prob_vars' are of a size 260, > counting the second index k for the first index i as well. Meaning that > while in 'finalcosts' for i=0, k=0:4 and for i=2, k=0:6 (which is what I > want), for the 'Prob_vars' decision variable index k=0:6 for both i=1 & > i=2. > > I'm fairly new to Python so I can't quite grasp where the problem lies. > > What I have tried: > > I've tried all combinations that I could think of for different > expressions of the 'Prob_vars' but nothing works properly. I also looked > anywhere I could think of online but I can't find an answer to this. From giannis.dafnomilis at gmail.com Tue Aug 7 13:38:44 2018 From: giannis.dafnomilis at gmail.com (giannis.dafnomilis at gmail.com) Date: Tue, 7 Aug 2018 10:38:44 -0700 (PDT) Subject: How do I create a variable where one index depends on the value of another index? In-Reply-To: References: Message-ID: Hey Peter. This worked like a charm! I can't believe I did not think of that, after wasting so many hours on it. Thank you so much for the help! From __peter__ at web.de Tue Aug 7 14:49:39 2018 From: __peter__ at web.de (Peter Otten) Date: Tue, 07 Aug 2018 20:49:39 +0200 Subject: How do I create a variable where one index depends on the value of another index? References: Message-ID: giannis.dafnomilis at gmail.com wrote: > Hey Peter. > > This worked like a charm! I can't believe I did not think of that, after > wasting so many hours on it. > > Thank you so much for the help! You can make these things easier to debug by breaking your code into small functions. If there were separate functions def make_finalcosts(...): ... # i is a local variable here def make_probvars(...): ... # i is not defined here and there's no global i you'd have seen a NameError exception similar to the one below: >>> def example(m, n): ... return [i + k for j in range(m) for k in range(n)] ... >>> example(1, 2) Traceback (most recent call last): File "", line 1, in File "", line 2, in example File "", line 2, in NameError: name 'i' is not defined Writing a function in such a way that the result only depends on the function's arguments allows for easy testing. From fetchinson at googlemail.com Tue Aug 7 18:07:08 2018 From: fetchinson at googlemail.com (Fetchinson .) Date: Wed, 8 Aug 2018 00:07:08 +0200 Subject: How to make python pick up my new-and-shiny openssl shared object Message-ID: The highest version of openssl available on my system is 1.0.0 which is not good enough for pip these days (or github for that matter). So I've installed 1.1.0 to a custom location /home/fetch/opt. But if I do import ssl ssl.OPENSSL_VERSION it still shows me that it is using the system default 1.0.0. How do I tell python to use /home/fetch/opt for the ssl module? Note that I have /home/fetch/opt as the first entry in LD_LIBRARY_PATH. Also, I know for a fact that I don't need to recompile python for this so please don't suggest "just recompile python with the new openssl library" as the solution :) By the way my python is 2.7.3. -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From christian at python.org Wed Aug 8 01:25:33 2018 From: christian at python.org (Christian Heimes) Date: Wed, 8 Aug 2018 07:25:33 +0200 Subject: How to make python pick up my new-and-shiny openssl shared object In-Reply-To: References: Message-ID: On 2018-08-08 00:07, Fetchinson . via Python-list wrote: > The highest version of openssl available on my system is 1.0.0 which > is not good enough for pip these days (or github for that matter). So > I've installed 1.1.0 to a custom location /home/fetch/opt. But if I do > > import ssl > ssl.OPENSSL_VERSION > > it still shows me that it is using the system default 1.0.0. How do I > tell python to use /home/fetch/opt for the ssl module? Note that I > have /home/fetch/opt as the first entry in LD_LIBRARY_PATH. Also, I > know for a fact that I don't need to recompile python for this so > please don't suggest "just recompile python with the new openssl > library" as the solution :) Hi, first of all, you need to use the library directory for LD_LIBRARY_PATH. It's the directory that contains libssl*.so, probably /home/fetch/opt/lib or /home/fetch/opt/lib64. You may also have to recompile Python yourself. OpenSSL 1.0.2 is not ABI-compatible with OpenSSL 1.0.0. In case you want to use OpenSSL 1.1.0, you must update to a more recent version of Python, too. OpenSSL 1.1.0 support was added in 2.7.13. Christian From fetchinson at googlemail.com Wed Aug 8 07:25:30 2018 From: fetchinson at googlemail.com (Fetchinson .) Date: Wed, 8 Aug 2018 13:25:30 +0200 Subject: How to make python pick up my new-and-shiny openssl shared object In-Reply-To: References: Message-ID: On 8/8/18, Christian Heimes wrote: > On 2018-08-08 00:07, Fetchinson . via Python-list wrote: >> The highest version of openssl available on my system is 1.0.0 which >> is not good enough for pip these days (or github for that matter). So >> I've installed 1.1.0 to a custom location /home/fetch/opt. But if I do >> >> import ssl >> ssl.OPENSSL_VERSION >> >> it still shows me that it is using the system default 1.0.0. How do I >> tell python to use /home/fetch/opt for the ssl module? Note that I >> have /home/fetch/opt as the first entry in LD_LIBRARY_PATH. Also, I >> know for a fact that I don't need to recompile python for this so >> please don't suggest "just recompile python with the new openssl >> library" as the solution :) > > Hi, > > first of all, you need to use the library directory for LD_LIBRARY_PATH. > It's the directory that contains libssl*.so, probably > /home/fetch/opt/lib or /home/fetch/opt/lib64. Yes, you are right, thanks. But it still doesn't work, most probably because of the reason you mention below. > You may also have to recompile Python yourself. OpenSSL 1.0.2 is not > ABI-compatible with OpenSSL 1.0.0. In case you want to use OpenSSL > 1.1.0, you must update to a more recent version of Python, too. OpenSSL > 1.1.0 support was added in 2.7.13. Ach, you are right again! I thought openssl was fully backward compatible, if not, then indeed I need to recompile, which I'll do now. Thanks a lot, Daniel > Christian > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From exarkun at twistedmatrix.com Wed Aug 8 09:07:49 2018 From: exarkun at twistedmatrix.com (Jean-Paul Calderone) Date: Wed, 8 Aug 2018 09:07:49 -0400 Subject: [ANN] txkube 0.3.0 Message-ID: Hello all, I'm pleased to announce a new release of txkube, a Twisted-based library for interacting with Kubernetes using the HTTP API. The big news for this release is support for Python 3.6. Also included is support for multiple configuration files in the KUBECONFIG environment variable which allows for better configuration management practices. Here is an example of txkube usage, taken from the README: from __future__ import print_function from twisted.internet.task import react from txkube import network_kubernetes_from_context @react def main(reactor): k8s = network_kubernetes_from_context(reactor, u"minikube") d = k8s.versioned_client() d.addCallback( lambda client: client.list(client.model.v1.Namespace) ) d.addCallback(print) return d You can download txkube from PyPI You can contribute to its development on GitHub . Thanks to Least Authority TFA GmbH for sponsoring this development and to Craig Rodrigues for his efforts on Python 3 porting work. Jean-Paul Calderone From rachel.selkirk at gmail.com Wed Aug 8 12:18:49 2018 From: rachel.selkirk at gmail.com (Rachel Selkirk) Date: Wed, 8 Aug 2018 09:18:49 -0700 Subject: Looking for feedback from developers... Message-ID: Hi Everyone! I?ve been working on a free product that I really think could help out this community. No one likes stopping development to fix a build that broke from changing upstream dependencies. I?ve created a tool to deal with this, and I?m looking for beta users and some feedback from actual developers. This tool is like a time machine for all the external things in your build that are not tracked under version control. If you have 2 minutes to spare, I?d love your help! I want to know if this would be useful to you. Help with the 2 min survey an sign up for our beta . Learn more here. Thank you! From jorge.conrado at cptec.inpe.br Wed Aug 8 15:17:03 2018 From: jorge.conrado at cptec.inpe.br (jorge.conrado at cptec.inpe.br) Date: Wed, 08 Aug 2018 16:17:03 -0300 Subject: ImportError: libpoppler.so.76 Message-ID: Hi, I downloade an example from: http://basemaptutorial.readthedocs.io/en/latest/clip.html from mpl_toolkits.basemap import Basemap from matplotlib.path import Path from matplotlib.patches import PathPatch import matplotlib.pyplot as plt from osgeo import gdal import numpy import shapefile fig = plt.figure() ax = fig.add_subplot(111) sf = shapefile.Reader("ne_10m_admin_0_countries") for shape_rec in sf.shapeRecords(): if shape_rec.record[3] == 'Andorra': vertices = [] codes = [] pts = shape_rec.shape.points prt = list(shape_rec.shape.parts) + [len(pts)] for i in range(len(prt) - 1): for j in range(prt[i], prt[i+1]): vertices.append((pts[j][0], pts[j][1])) codes += [Path.MOVETO] codes += [Path.LINETO] * (prt[i+1] - prt[i] -2) codes += [Path.CLOSEPOLY] clip = Path(vertices, codes) clip = PathPatch(clip, transform=ax.transData) m = Basemap(llcrnrlon=1.4, llcrnrlat=42.4, urcrnrlon=1.77, urcrnrlat=42.7, resolution = None, projection = 'cyl') ds = gdal.Open('srtm_37_04.tif') data = ds.ReadAsArray() gt = ds.GetGeoTransform() x = numpy.linspace(gt[0], gt[0] + gt[1] * data.shape[1], data.shape[1]) y = numpy.linspace(gt[3], gt[3] + gt[5] * data.shape[0], data.shape[0]) xx, yy = numpy.meshgrid(x, y) cs = m.contourf(xx,yy,data,range(0, 3600, 200)) for contour in cs.collections: contour.set_clip_path(clip) plt.show() Then I did. python3 for this script and I had: python mascara.py Traceback (most recent call last): File "mascara.py", line 5, in from osgeo import gdal File "/home/conrado/anaconda3/lib/python2.7/site-packages/osgeo/__init__.py", line 21, in _gdal = swig_import_helper() File "/home/conrado/anaconda3/lib/python2.7/site-packages/osgeo/__init__.py", line 17, in swig_import_helper _mod = imp.load_module('_gdal', fp, pathname, description) ImportError: libpoppler.so.76: cannot open shared object file: No such file or directory Please, what can I do to solve this. Thanks, Conrado From mausg at mail.com Wed Aug 8 17:03:58 2018 From: mausg at mail.com (mausg at mail.com) Date: 8 Aug 2018 21:03:58 GMT Subject: NLTK References: Message-ID: On 2018-08-07, Stefan Ram wrote: > Steven D'Aprano writes: >>In natural language, words are more complicated than just space-separated >>units. Some languages don't use spaces as a word delimiter. > > Even above, the word ?units? is neither directly preceded > nor directly followed by a space. > > In the end, one can make an arbitrary choice about where one > wants to place the border between syntax and morphology. > > For the case of English, I can define a word to be a > sequence of letters (including the apostrophe), that is > sorrounded by non-letter characters. > >>Recognising open compound words is difficult. "Real estate" is an open >>compound word, but "real cheese" and "my estate" are both two words. > > This is just a part of the more general problem to parse and > interpret a sentence. It is not more difficult than the > interpretation of other pairs of words in a sentence. > >>Another problem for English speakers is deciding whether to treat >>contractions as a single word, or split them? >>"don't" --> "do" "n't" >>"they'll" --> "they" "'ll" > > They are a single word by my definition. But this is just > the surface of the input. The input could be translated into > a "deep-structure" intermediate language that than splits > some source words into several units or joins some source > words into a single unit. > >>Punctuation marks should either be stripped out of sentences before >>splitting into words, or treated as distinct tokens. We don't want >>"tokens" and "tokens." to be treated as distinct words, just because one >>happened to fall at the end of a sentence and one didn't. > > Yes, but this is quite trivial compared to the problem > of parsing and interpreting a natural-language sentence. > Thanks all for the replies. It seems that I do not really need NLTK. split() will do me. Again Thanks -- Maus at ireland.com Will Rant For Food From python at mrabarnett.plus.com Wed Aug 8 18:16:33 2018 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 8 Aug 2018 23:16:33 +0100 Subject: ImportError: libpoppler.so.76 In-Reply-To: References: Message-ID: On 2018-08-08 20:17, jorge.conrado at cptec.inpe.br wrote: > > > Hi, > > I downloade an example from: > > > http://basemaptutorial.readthedocs.io/en/latest/clip.html > > > from mpl_toolkits.basemap import Basemap > from matplotlib.path import Path > from matplotlib.patches import PathPatch > import matplotlib.pyplot as plt > from osgeo import gdal > import numpy > import shapefile > > fig = plt.figure() > ax = fig.add_subplot(111) > > sf = shapefile.Reader("ne_10m_admin_0_countries") > > for shape_rec in sf.shapeRecords(): > if shape_rec.record[3] == 'Andorra': > vertices = [] > codes = [] > pts = shape_rec.shape.points > prt = list(shape_rec.shape.parts) + [len(pts)] > for i in range(len(prt) - 1): > for j in range(prt[i], prt[i+1]): > vertices.append((pts[j][0], pts[j][1])) > codes += [Path.MOVETO] > codes += [Path.LINETO] * (prt[i+1] - prt[i] -2) > codes += [Path.CLOSEPOLY] > clip = Path(vertices, codes) > clip = PathPatch(clip, transform=ax.transData) > > > m = Basemap(llcrnrlon=1.4, > llcrnrlat=42.4, > urcrnrlon=1.77, > urcrnrlat=42.7, > resolution = None, > projection = 'cyl') > > ds = gdal.Open('srtm_37_04.tif') > data = ds.ReadAsArray() > > gt = ds.GetGeoTransform() > x = numpy.linspace(gt[0], gt[0] + gt[1] * data.shape[1], data.shape[1]) > y = numpy.linspace(gt[3], gt[3] + gt[5] * data.shape[0], data.shape[0]) > > xx, yy = numpy.meshgrid(x, y) > > cs = m.contourf(xx,yy,data,range(0, 3600, 200)) > > for contour in cs.collections: > contour.set_clip_path(clip) > > plt.show() > > > > Then I did. > > python3 for this script and I had: > > > > > python mascara.py > Traceback (most recent call last): > File "mascara.py", line 5, in > from osgeo import gdal > File > "/home/conrado/anaconda3/lib/python2.7/site-packages/osgeo/__init__.py", > line 21, in > _gdal = swig_import_helper() > File > "/home/conrado/anaconda3/lib/python2.7/site-packages/osgeo/__init__.py", > line 17, in swig_import_helper > _mod = imp.load_module('_gdal', fp, pathname, description) > ImportError: libpoppler.so.76: cannot open shared object file: No such > file or directory > > > > Please, what can I do to solve this. > You said you did python3, but the traceback mentions python2.7, which means that you have Anaconda for Python 2.7, not Anaconda for Python 3. Have you tried it with python (for Python 2) instead? From tjol at tjol.eu Wed Aug 8 18:16:45 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 9 Aug 2018 00:16:45 +0200 Subject: Non-unicode file names Message-ID: On *nix, file names are bytes. In real life, we prefer to think of file names as strings. How non-ASCII file names are created is determined by the locale, and on most systems these days, every locale uses UTF-8 and everybody's happy. Of course this doesn't mean you'll never run into and old directory tree from the pre-UTF8 age using some other encoding, and it doesn't prevent people from doing silly things in file names. Python deals with this tolerably well: by convention, file names are strings, but you can use bytes for file names if you wish. The docs [1] warn you about the situation. [1] https://docs.python.org/3/library/os.path.html If Python runs into a non-UTF8 (better: non-decodable) file name and has to return a str, it uses surrogate escape codes. So far so good. Right? This leads to the unfortunate situation that you can't always print() file names, as print() is strict and refuses to toy with surrogates. To be more explicit, the script print(__file__) will fail depending on the file name. This feels wrong... (though every bit of behaviour is correct) (The situation can't arise on Windows, and Python 2 will pretend nothing happened in true UNIX style) Demo script to try at home below. -- Thomas # -*- coding: UTF-8 -*- from __future__ import unicode_literals, print_function import sys import os.path import subprocess import tempfile import shutil script = 'print(__file__)\n' file_names = ['?.py', '?.py', '?.py'.encode('latin9')] PY = sys.executable tmpdir = tempfile.mkdtemp() for fn in file_names: if isinstance(fn, bytes): path = os.path.join(tmpdir.encode('ascii'), fn) else: path = os.path.join(tmpdir, fn) print('? creating', path) with open(path, 'w') as fp: fp.write(script) print('? running', PY, path) status = subprocess.call([PY, path]) print('? exited with status', status) print('? cleaning up') shutil.rmtree(tmpdir) # End of script ####################################################################### # Output from Python 3.6.5 on Linux (Ubuntu 18.04):: # # ? creating /tmp/tmp_a4h5n22/?.py # ? running /usr/bin/python3 /tmp/tmp_a4h5n22/?.py # /tmp/tmp_a4h5n22/?.py # ? exited with status 0 # ? creating /tmp/tmp_a4h5n22/?.py # ? running /usr/bin/python3 /tmp/tmp_a4h5n22/?.py # /tmp/tmp_a4h5n22/?.py # ? exited with status 0 # ? creating b'/tmp/tmp_a4h5n22/\xa4.py' # ? running /usr/bin/python3 b'/tmp/tmp_a4h5n22/\xa4.py' # Traceback (most recent call last): # File "/tmp/tmp_a4h5n22/\udca4.py", line 1, in # print(__file__) # UnicodeEncodeError: 'utf-8' codec can't encode character '\udca4' in position 17: surrogates not allowed # ? exited with status 1 # ? cleaning up # # Python 2.7.15rc1 on Linux (Ubuntu): # # ? creating /tmp/tmp_U_LPp/?.py # ? running /usr/bin/python2 /tmp/tmp_U_LPp/?.py # /tmp/tmp_U_LPp/?.py # ? exited with status 0 # ? creating /tmp/tmp_U_LPp/?.py # ? running /usr/bin/python2 /tmp/tmp_U_LPp/?.py # /tmp/tmp_U_LPp/?.py # ? exited with status 0 # ? creating /tmp/tmp_U_LPp/?.py # ? running /usr/bin/python2 /tmp/tmp_U_LPp/?.py # /tmp/tmp_U_LPp/?.py # ? exited with status 0 # ? cleaning up # # Python 3.7.0 on Windows 10:: # # ? creating C:\Users\tjol\AppData\Local\Temp\tmpzprwnyc2\?.py # ? running C:\Users\tjol\AppData\Local\Programs\Python\Python37\python.exe C:\Users\tjol\AppData\Local\Temp\tmpzprwnyc2\? # ?.py # C:\Users\tjol\AppData\Local\Temp\tmpzprwnyc2\?.py # ? exited with status 0 # ? creating C:\Users\tjol\AppData\Local\Temp\tmpzprwnyc2\?.py # ? running C:\Users\tjol\AppData\Local\Programs\Python\Python37\python.exe C:\Users\tjol\AppData\Local\Temp\tmpzprwnyc2\? # .py # C:\Users\tjol\AppData\Local\Temp\tmpzprwnyc2\?.py # ? exited with status 0 # ? creating b'C:\\Users\\tjol\\AppData\\Local\\Temp\\tmpzprwnyc2\\\xa4.py' # Traceback (most recent call last): # File ".\bytes_file_names2.py", line 25, in # with open(path, 'w') as fp: # UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa4 in position 45: invalid start byte # # Python 2.7.15 on Windows 10: # # Traceback (most recent call last): # File ".\bytes_file_names2.py", line 24, in # print('??? creating', path) # File "C:\Python27\lib\encodings\cp850.py", line 12, in encode # return codecs.charmap_encode(input,errors,encoding_map) # UnicodeEncodeError: 'charmap' codec can't encode character u'\u25ba' in position 0: character maps to From cs at cskk.id.au Wed Aug 8 19:26:06 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 9 Aug 2018 09:26:06 +1000 Subject: QOTW! (was: NLTK) In-Reply-To: References: Message-ID: <20180808232606.GA30291@cskk.homeip.net> On 08Aug2018 21:03, mausg at mail.com wrote: >[...] It seems that I do not really need NLTK. split() will do me. [...] +1 QOTW Cheers, Cameron Simpson From python at mrabarnett.plus.com Wed Aug 8 19:48:34 2018 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 9 Aug 2018 00:48:34 +0100 Subject: Non-unicode file names In-Reply-To: References: Message-ID: <04cb413e-3d1e-7001-673f-9dca94173980@mrabarnett.plus.com> On 2018-08-08 23:16, Thomas Jollans wrote: > On *nix, file names are bytes. In real life, we prefer to think of file > names as strings. How non-ASCII file names are created is determined by > the locale, and on most systems these days, every locale uses UTF-8 and > everybody's happy. Of course this doesn't mean you'll never run into and > old directory tree from the pre-UTF8 age using some other encoding, and > it doesn't prevent people from doing silly things in file names. > > Python deals with this tolerably well: by convention, file names are > strings, but you can use bytes for file names if you wish. The docs [1] > warn you about the situation. > > [1] https://docs.python.org/3/library/os.path.html > > If Python runs into a non-UTF8 (better: non-decodable) file name and has > to return a str, it uses surrogate escape codes. So far so good. Right? > > This leads to the unfortunate situation that you can't always print() > file names, as print() is strict and refuses to toy with surrogates. > > To be more explicit, the script > > print(__file__) > > will fail depending on the file name. This feels wrong... (though every > bit of behaviour is correct) > > (The situation can't arise on Windows, and Python 2 will pretend nothing > happened in true UNIX style) > > Demo script to try at home below. > [snip] Is it true that Unix filenames can contain control characters, e.g. \x07? When happens when you print them out? I think it's not just a problem with surrogate escapes. From tjol at tjol.eu Wed Aug 8 20:14:33 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 9 Aug 2018 02:14:33 +0200 Subject: Non-unicode file names In-Reply-To: <04cb413e-3d1e-7001-673f-9dca94173980@mrabarnett.plus.com> References: <04cb413e-3d1e-7001-673f-9dca94173980@mrabarnett.plus.com> Message-ID: On 09/08/18 01:48, MRAB wrote: > On 2018-08-08 23:16, Thomas Jollans wrote: >> On *nix, file names are bytes. In real life, we prefer to think of file >> names as strings. How non-ASCII file names are created is determined by >> the locale, and on most systems these days, every locale uses UTF-8 and >> everybody's happy. Of course this doesn't mean you'll never run into and >> old directory tree from the pre-UTF8 age using some other encoding, and >> it doesn't prevent people from doing silly things in file names. >> >> Python deals with this tolerably well: by convention, file names are >> strings, but you can use bytes for file names if you wish. The docs [1] >> warn you about the situation. >> >> [1] https://docs.python.org/3/library/os.path.html >> >> If Python runs into a non-UTF8 (better: non-decodable) file name and has >> to return a str, it uses surrogate escape codes. So far so good. Right? >> >> This leads to the unfortunate situation that you can't always print() >> file names, as print() is strict and refuses to toy with surrogates. >> >> To be more explicit, the script >> >> ???? print(__file__) >> >> will fail depending on the file name. This feels wrong... (though every >> bit of behaviour is correct) >> >> (The situation can't arise on Windows, and Python 2 will pretend nothing >> happened in true UNIX style) >> >> Demo script to try at home below. >> > [snip] > > Is it true that Unix filenames can contain control characters, e.g. \x07? > > When happens when you print them out? > > I think it's not just a problem with surrogate escapes. Not a problem (or: not an exception), as those are ASCII and thus UTF-8. Python 3.6.5 (default, Apr 1 2018, 05:46:30) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> with open('\x07.py', 'w') as fp: ... fp.write('print(__file__)\n') ... 16 >>> import sys; import subprocess >>> subprocess.call([sys.executable, '\x07.py']) .py 0 >>> As you might expect, it beeped when printing '\x07.py' (and showed .py) From python at mrabarnett.plus.com Wed Aug 8 22:14:03 2018 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 9 Aug 2018 03:14:03 +0100 Subject: Non-unicode file names In-Reply-To: References: <04cb413e-3d1e-7001-673f-9dca94173980@mrabarnett.plus.com> Message-ID: On 2018-08-09 01:14, Thomas Jollans wrote: > On 09/08/18 01:48, MRAB wrote: >> On 2018-08-08 23:16, Thomas Jollans wrote: >>> On *nix, file names are bytes. In real life, we prefer to think of file >>> names as strings. How non-ASCII file names are created is determined by >>> the locale, and on most systems these days, every locale uses UTF-8 and >>> everybody's happy. Of course this doesn't mean you'll never run into and >>> old directory tree from the pre-UTF8 age using some other encoding, and >>> it doesn't prevent people from doing silly things in file names. >>> >>> Python deals with this tolerably well: by convention, file names are >>> strings, but you can use bytes for file names if you wish. The docs [1] >>> warn you about the situation. >>> >>> [1] https://docs.python.org/3/library/os.path.html >>> >>> If Python runs into a non-UTF8 (better: non-decodable) file name and has >>> to return a str, it uses surrogate escape codes. So far so good. Right? >>> >>> This leads to the unfortunate situation that you can't always print() >>> file names, as print() is strict and refuses to toy with surrogates. >>> >>> To be more explicit, the script >>> >>> ???? print(__file__) >>> >>> will fail depending on the file name. This feels wrong... (though every >>> bit of behaviour is correct) >>> >>> (The situation can't arise on Windows, and Python 2 will pretend nothing >>> happened in true UNIX style) >>> >>> Demo script to try at home below. >>> >> [snip] >> >> Is it true that Unix filenames can contain control characters, e.g. \x07? >> >> When happens when you print them out? >> >> I think it's not just a problem with surrogate escapes. > > Not a problem (or: not an exception), as those are ASCII and thus UTF-8. > > Python 3.6.5 (default, Apr 1 2018, 05:46:30) > [GCC 7.3.0] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> with open('\x07.py', 'w') as fp: > ... fp.write('print(__file__)\n') > ... > 16 >>>> import sys; import subprocess >>>> subprocess.call([sys.executable, '\x07.py']) > .py > 0 >>>> > > As you might expect, it beeped when printing '\x07.py' (and showed .py) > And that's OK, is it? :-) From cs at cskk.id.au Wed Aug 8 22:39:49 2018 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 9 Aug 2018 12:39:49 +1000 Subject: Non-unicode file names In-Reply-To: References: Message-ID: <20180809023949.GA55895@cskk.homeip.net> On 09Aug2018 03:14, MRAB wrote: [...] >>>Is it true that Unix filenames can contain control characters, e.g. \x07? Yep. They're just byte strings. You can't have \0 (NUL) because the API uses NUL terminated strings, and you can't use slash '/' in the filename components because that is the component separator. But otherwise you can basicly use anything - the OS itself doesn't care. There are some (platform dependent) length limits, and the underlying mounted filesystem you're accessing may itself have special rules (eg nonUNIX filesystems like FAT32, etc). >>>When happens when you print them out? They get written out? If you're printing to a terminal of some kind then it will do whatever the bytes from the filename tell it to, as that's what terminals do. >>>>>import sys; import subprocess >>>>>subprocess.call([sys.executable, '\x07.py']) >>.py >>0 >>>>> >> >>As you might expect, it beeped when printing '\x07.py' (and showed .py) >> >And that's OK, is it? :-) Of course it is :-) \07 is the ASCII BEL character, so it rings the terminal's bell. Modern software terminals emulate that to a better or worse degree. Suppose you're verbally reciting a filename (or, of course, printing the filename to a voder). Only Victor Borge will provide a full verbal pronunciation of things [1] [1] https://www.youtube.com/results?search_query=victor+gorge+phonetic+punctuation Cheers, Cameron Simpson From songofacandy at gmail.com Wed Aug 8 23:13:46 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Thu, 9 Aug 2018 12:13:46 +0900 Subject: Non-unicode file names In-Reply-To: References: Message-ID: Please use Python 3.7. Python 3.7 has several improvements on this area. * When PEP 538 or 540 is used, default error handler for stdio is surrogateescape * You can sys.stdout.reconfigure(errors='surrogateescape') For Python 3.6, I think best way to allow arbitrary bytes on stdout is using `PYTHONIOENCODING=utf-8:surrogateescape` environment variable. Regards, From marko at pacujo.net Thu Aug 9 02:06:04 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 09 Aug 2018 09:06:04 +0300 Subject: Non-unicode file names References: Message-ID: <87wot0hy5v.fsf@elektro.pacujo.net> INADA Naoki : > For Python 3.6, I think best way to allow arbitrary bytes on stdout is > using `PYTHONIOENCODING=utf-8:surrogateescape` environment variable. Good info! Marko From tjol at tjol.eu Thu Aug 9 04:10:32 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 9 Aug 2018 10:10:32 +0200 Subject: Non-unicode file names In-Reply-To: References: Message-ID: On 09/08/18 05:13, INADA Naoki wrote: > Please use Python 3.7. > > Python 3.7 has several improvements on this area. Thanks! Darkly remembering something about UTF-8 mode, I suspected it might... > > * When PEP 538 or 540 is used, default error handler for stdio is > surrogateescape > * You can sys.stdout.reconfigure(errors='surrogateescape') > > For Python 3.6, I think best way to allow arbitrary bytes on stdout is using > `PYTHONIOENCODING=utf-8:surrogateescape` environment variable. From arj.python at gmail.com Thu Aug 9 04:13:07 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Thu, 9 Aug 2018 12:13:07 +0400 Subject: using python's logo in your own logo Message-ID: in short, can you use python's logo in your own logo without credit? yours, Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius From vishalhule24 at gmail.com Thu Aug 9 04:17:58 2018 From: vishalhule24 at gmail.com (vishalhule24 at gmail.com) Date: Thu, 9 Aug 2018 01:17:58 -0700 (PDT) Subject: Python MySQL Guide Message-ID: <0fbeeedc-178d-40c1-abce-5be84e7aa66d@googlegroups.com> Refer this complete guide on working with Python and MySQL https://pynative.com/python-mysql-tutorial/ From tjol at tjol.eu Thu Aug 9 04:24:20 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 9 Aug 2018 10:24:20 +0200 Subject: using python's logo in your own logo In-Reply-To: References: Message-ID: <4285f4c1-cc42-dd12-62ca-44583d4c65fd@tjol.eu> On 09/08/18 10:13, Abdur-Rahmaan Janhangeer wrote: > in short, > > can you use python's logo in your own logo without credit? > > yours, > > Abdur-Rahmaan Janhangeer > https://github.com/Abdur-rahmaanJ > Mauritius > See https://www.python.org/community/logos/ and https://www.python.org/psf/trademarks/ From alister.ware at ntlworld.com Thu Aug 9 06:38:06 2018 From: alister.ware at ntlworld.com (Alister) Date: Thu, 09 Aug 2018 10:38:06 GMT Subject: using python's logo in your own logo Message-ID: On Thu, 09 Aug 2018 12:13:07 +0400, Abdur-Rahmaan Janhangeer wrote: > in short, > > can you use python's logo in your own logo without credit? > > yours, > > Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius I would expect the answer to that to be a big NO. I am pretty sure you would need to get permission. -- VICARIOUSLY experience some reason to LIVE!! From rosuav at gmail.com Thu Aug 9 07:57:34 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 9 Aug 2018 21:57:34 +1000 Subject: using python's logo in your own logo In-Reply-To: References: Message-ID: On Thu, Aug 9, 2018 at 8:38 PM, Alister via Python-list wrote: > On Thu, 09 Aug 2018 12:13:07 +0400, Abdur-Rahmaan Janhangeer wrote: > >> in short, >> >> can you use python's logo in your own logo without credit? >> >> yours, >> >> Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius > > I would expect the answer to that to be a big NO. > I am pretty sure you would need to get permission. Not quite! If you check the links that Thomas posted, you'll see that there are a number of uses that don't require permission. Or rather, uses for which blanket permission has been granted. https://www.python.org/psf/trademarks/ ChrisA From iranna.gani28 at gmail.com Thu Aug 9 08:11:01 2018 From: iranna.gani28 at gmail.com (Iranna Mathapati) Date: Thu, 9 Aug 2018 17:41:01 +0530 Subject: failed to execute scapy commands from remote server Message-ID: Hi Team, I have to ssh a remote server and execute scapy commands there. but its able to execute all linux commands from remote server handle I use pexpect and hitting issue. 1. SSH to remote shell via pexpect from current server parser_ip_add='192.168.1.83' child_remote = pexpect.spawn('/usr/bin/ssh',['AST@ {0}'.format(parser_ip_add)]) ssh_newkey = 'Are you sure you want to continue connecting' i=child_remote.expect([pexpect.TIMEOUT, ssh_newkey, 'password:']) if i==2: child_remote.sendline('XXXXX') <<< password child_remote.sendline('sudo su') child_remote.expect('password for AST: ') child_remote.sendline('XXXXX') <<<<>> child_remote.sendline=sendp(arp3,iface='enp6s0f0',count=100) Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/site-packages/scapy/sendrecv.py", line 256, in sendp __gen_send(conf.L2socket(iface=iface, *args, **kargs), x, inter=inter, loop=loop, count=count, verbose=verbose, realtime=realtime) File "/usr/local/lib/python2.7/site-packages/scapy/arch/linux.py", line 425, in __init__ self.ins.bind((iface, type)) File "/usr/local/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 19] No such device >>> child_remote.sendline(sendp(arp3,iface='enp6s0f0',count=100)) KeyboardInterrupt From rhodri at kynesim.co.uk Thu Aug 9 09:02:54 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 9 Aug 2018 14:02:54 +0100 Subject: failed to execute scapy commands from remote server In-Reply-To: References: Message-ID: <8f47441f-73aa-47e0-6dfa-7d55ad8b7e1f@kynesim.co.uk> On 09/08/18 13:11, Iranna Mathapati wrote: > 2. When I try to execute scapy cmds in remote server I get the below > error > >>>> child_remote.sendline=sendp(arp3,iface='enp6s0f0',count=100) > Traceback (most recent call last): > > File "", line 1, in > > File "/usr/local/lib/python2.7/site-packages/scapy/sendrecv.py", line > 256, in sendp > > __gen_send(conf.L2socket(iface=iface, *args, **kargs), x, inter=inter, > loop=loop, count=count, verbose=verbose, realtime=realtime) > > File "/usr/local/lib/python2.7/site-packages/scapy/arch/linux.py", line > 425, in __init__ > > self.ins.bind((iface, type)) > > File "/usr/local/lib/python2.7/socket.py", line 224, in meth > > return getattr(self._sock,name)(*args) > > socket.error: [Errno 19] No such device At a guess, you are asking to send on an interface that doesn't exist. Does your machine have a network interface called "enp6s0f0"? -- Rhodri James *-* Kynesim Ltd From xuanwu348 at 163.com Thu Aug 9 11:16:37 2018 From: xuanwu348 at 163.com (xuanwu348) Date: Thu, 9 Aug 2018 23:16:37 +0800 (CST) Subject: name 'aLOCK' is not defined When I add aLOCK = threading.RLock() behind if __name__ == "__main__" Message-ID: <639b7d96.83da.1651f42d0e1.Coremail.xuanwu348@163.com> Hi team Good day The problem I meet when I add "aLOCK = threading.RLock()" to PositionB, the program will report error "name 'aLOCK' is not defined ", but when I change this code to PositionA, it can run normally, is there any difference for the code between 'if __name__ == "__main__:"', can you help me, thanks! The file I was attached, please change the extend ".pyx" to ".py", thanks, code as below, tried python2.7 and python3.4: import threading import time from multiprocessing import Process #PositionA aLOCK = threading.RLock() def Save_runnedCMD(filename, strings): aLOCK.acquire() with open(filename, "at") as f: f.write(str(strings) + "\n\r") aLOCK.release() def runCMD(filename): time.sleep(1) cmd = "testtest" Save_runnedCMD(filename, cmd) def Thr_run(filename): t = [] for i in range(2): tt = threading.Thread(target = runCMD, args=(filename,)) tt.start() t.append(tt) for tt in t: tt.join() if __name__ == "__main__": filename = "./testaaaaa.log" #PositionB #aLOCK = threading.RLock() while 1: t1 = Process(target=Thr_run, args=(filename, )) t1.start() t1.join() Error info as below: D:\Appppp>python testa.py Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner self.run() File "C:\Python34\lib\threading.py", line 869, in run self._target(*self._args, **self._kwargs) File "D:\Appppp\testa.py", line 15, in runCMD Save_runnedCMD(filename, cmd) File "D:\Appppp\testa.py", line 7, in Save_runnedCMD aLOCK.acquire() NameError: name 'aLOCK' is not defined Exception in thread Thread-2: Traceback (most recent call last): File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner self.run() File "C:\Python34\lib\threading.py", line 869, in run self._target(*self._args, **self._kwargs) File "D:\Appppp\testa.py", line 15, in runCMD Save_runnedCMD(filename, cmd) File "D:\Appppp\testa.py", line 7, in Save_runnedCMD aLOCK.acquire() NameError: name 'aLOCK' is not defined From cseberino at gmail.com Thu Aug 9 11:30:56 2018 From: cseberino at gmail.com (cseberino at gmail.com) Date: Thu, 9 Aug 2018 08:30:56 -0700 (PDT) Subject: Why this regex for string literals can't handle escaped quotes?.... '"(\\.|[^"])*"' Message-ID: <896892c5-aeff-4402-a7b2-1ba56261336d@googlegroups.com> Why this regex for string literals can't handle escaped quotes?.... '"(\\.|[^"])*"' See this... >>> string_re = '"(\\.|[^"])*"' >>> re.match(string_re, '"aaaa"') <_sre.SRE_Match object; span=(0, 6), match='"aaaa"'> >>> re.match(string_re, '"aa\"aa"') <_sre.SRE_Match object; span=(0, 4), match='"aa"'> How make the last match be the entire string '"aa\"aa"' ? cs From Karsten.Hilbert at gmx.net Thu Aug 9 11:51:45 2018 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Thu, 9 Aug 2018 17:51:45 +0200 Subject: name 'aLOCK' is not defined When I add aLOCK = threading.RLock() behind if __name__ == "__main__" In-Reply-To: <639b7d96.83da.1651f42d0e1.Coremail.xuanwu348@163.com> References: <639b7d96.83da.1651f42d0e1.Coremail.xuanwu348@163.com> Message-ID: <20180809155145.GG3862@hermes.hilbert.loc> On Thu, Aug 09, 2018 at 11:16:37PM +0800, xuanwu348 wrote: > The problem I meet when I add "aLOCK = threading.RLock()" to PositionB, the program will report error "name 'aLOCK' is not defined ", You mean to say: *move*, not *add*. That is why aLock is out of scope. ..../python3-doc/html/tutorial/classes.html#python-scopes-and-namespaces > but when I change this code to PositionA, it can run normally, is there any difference for the code between 'if __name__ == "__main__:"', can you help me, thanks! > > The file I was attached, please change the extend ".pyx" to ".py", thanks, code as below, tried python2.7 and python3.4: > > import threading > import time > from multiprocessing import Process > > #PositionA > aLOCK = threading.RLock() > > def Save_runnedCMD(filename, strings): > aLOCK.acquire() > with open(filename, "at") as f: > f.write(str(strings) + "\n\r") > aLOCK.release() > > def runCMD(filename): > time.sleep(1) > cmd = "testtest" > Save_runnedCMD(filename, cmd) > > > def Thr_run(filename): > t = [] > for i in range(2): > tt = threading.Thread(target = runCMD, args=(filename,)) > tt.start() > t.append(tt) > for tt in t: > tt.join() > > if __name__ == "__main__": > filename = "./testaaaaa.log" > > #PositionB > #aLOCK = threading.RLock() > > while 1: > t1 = Process(target=Thr_run, args=(filename, )) > t1.start() > t1.join() > > Error info as below: > D:\Appppp>python testa.py > Exception in thread Thread-1: > Traceback (most recent call last): > File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner > self.run() > File "C:\Python34\lib\threading.py", line 869, in run > self._target(*self._args, **self._kwargs) > File "D:\Appppp\testa.py", line 15, in runCMD > Save_runnedCMD(filename, cmd) > File "D:\Appppp\testa.py", line 7, in Save_runnedCMD > aLOCK.acquire() > NameError: name 'aLOCK' is not defined > > Exception in thread Thread-2: > Traceback (most recent call last): > File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner > self.run() > File "C:\Python34\lib\threading.py", line 869, in run > self._target(*self._args, **self._kwargs) > File "D:\Appppp\testa.py", line 15, in runCMD > Save_runnedCMD(filename, cmd) > File "D:\Appppp\testa.py", line 7, in Save_runnedCMD > aLOCK.acquire() > NameError: name 'aLOCK' is not defined > -- > https://mail.python.org/mailman/listinfo/python-list -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B From applemask at gmail.com Thu Aug 9 12:09:22 2018 From: applemask at gmail.com (Apple) Date: Thu, 9 Aug 2018 09:09:22 -0700 (PDT) Subject: Embedded Python and multiprocessing on Windows? Message-ID: I've been working on a project involving embedding Python into a Windows application. I've got all of that working fine on the C++ side, but the script side seems to be hitting a dead end with multiprocessing. When my script tries to run the same multiprocessing code that works in a non-embedded environment, the code doesn't appear to be executed at all. A StackOverflow thread gave me a few ideas to try out, so I gave it a shot with these lines added: sys.argv = ['C:\\path\\to\\script.py'] mp.set_executable("C:\\Python37\\python.exe") p = mp.Process(target=somefunction) p.start() Still no joy. However, a Python.exe window does pop up for a tenth of a second, so *something* is happening. So does multiprocessing actually work in embedded Python, or am I at a dead end here? My guess is that it's probably linked to Windows not having fork, so the combination of being embedded and having to spawn is making things go goofy. From larry.martell at gmail.com Thu Aug 9 12:12:07 2018 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 9 Aug 2018 12:12:07 -0400 Subject: tabs vs. spaces Message-ID: https://www.youtube.com/watch?v=SsoOG6ZeyUI&feature=youtu.be From xuanwu348 at 163.com Thu Aug 9 12:24:25 2018 From: xuanwu348 at 163.com (xuanwu348) Date: Fri, 10 Aug 2018 00:24:25 +0800 (CST) Subject: name 'aLOCK' is not defined When I add aLOCK = threading.RLock() behind if __name__ == "__main__" In-Reply-To: <20180809155145.GG3862@hermes.hilbert.loc> References: <639b7d96.83da.1651f42d0e1.Coremail.xuanwu348@163.com> <20180809155145.GG3862@hermes.hilbert.loc> Message-ID: <231b0eff.8721.1651f80e3a8.Coremail.xuanwu348@163.com> Thanks for your reply! Yes, move the code from positionA(can run normally) to positionB(exception with name undefined) I find this content "https://docs.python.org/3.3/tutorial/classes.html#python-scopes-and-namespaces" But I still don't undewrstand the differenct of scopes-and-namespaces between positionA and positionB, I think the variable "aLock" at these positionA or positionB are all global. The function can find threading.RLock() by name ""aLock". At 2018-08-09 23:51:45, "Karsten Hilbert" wrote: >On Thu, Aug 09, 2018 at 11:16:37PM +0800, xuanwu348 wrote: > >> The problem I meet when I add "aLOCK = threading.RLock()" to PositionB, the program will report error "name 'aLOCK' is not defined ", > >You mean to say: *move*, not *add*. > >That is why aLock is out of scope. > > ..../python3-doc/html/tutorial/classes.html#python-scopes-and-namespaces > >> but when I change this code to PositionA, it can run normally, is there any difference for the code between 'if __name__ == "__main__:"', can you help me, thanks! >> >> The file I was attached, please change the extend ".pyx" to ".py", thanks, code as below, tried python2.7 and python3.4: >> >> import threading >> import time >> from multiprocessing import Process >> >> #PositionA >> aLOCK = threading.RLock() >> >> def Save_runnedCMD(filename, strings): >> aLOCK.acquire() >> with open(filename, "at") as f: >> f.write(str(strings) + "\n\r") >> aLOCK.release() >> >> def runCMD(filename): >> time.sleep(1) >> cmd = "testtest" >> Save_runnedCMD(filename, cmd) >> >> >> def Thr_run(filename): >> t = [] >> for i in range(2): >> tt = threading.Thread(target = runCMD, args=(filename,)) >> tt.start() >> t.append(tt) >> for tt in t: >> tt.join() >> >> if __name__ == "__main__": >> filename = "./testaaaaa.log" >> >> #PositionB >> #aLOCK = threading.RLock() >> >> while 1: >> t1 = Process(target=Thr_run, args=(filename, )) >> t1.start() >> t1.join() >> >> Error info as below: >> D:\Appppp>python testa.py >> Exception in thread Thread-1: >> Traceback (most recent call last): >> File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner >> self.run() >> File "C:\Python34\lib\threading.py", line 869, in run >> self._target(*self._args, **self._kwargs) >> File "D:\Appppp\testa.py", line 15, in runCMD >> Save_runnedCMD(filename, cmd) >> File "D:\Appppp\testa.py", line 7, in Save_runnedCMD >> aLOCK.acquire() >> NameError: name 'aLOCK' is not defined >> >> Exception in thread Thread-2: >> Traceback (most recent call last): >> File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner >> self.run() >> File "C:\Python34\lib\threading.py", line 869, in run >> self._target(*self._args, **self._kwargs) >> File "D:\Appppp\testa.py", line 15, in runCMD >> Save_runnedCMD(filename, cmd) >> File "D:\Appppp\testa.py", line 7, in Save_runnedCMD >> aLOCK.acquire() >> NameError: name 'aLOCK' is not defined >> -- >> https://mail.python.org/mailman/listinfo/python-list > >-- >GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B >-- >https://mail.python.org/mailman/listinfo/python-list From __peter__ at web.de Thu Aug 9 13:03:29 2018 From: __peter__ at web.de (Peter Otten) Date: Thu, 09 Aug 2018 19:03:29 +0200 Subject: Why this regex for string literals can't handle escaped quotes?.... '"(\\.|[^"])*"' References: <896892c5-aeff-4402-a7b2-1ba56261336d@googlegroups.com> Message-ID: cseberino at gmail.com wrote: > Why this regex for string literals > can't handle escaped quotes?.... '"(\\.|[^"])*"' > > See this... > >>>> string_re = '"(\\.|[^"])*"' > >>>> re.match(string_re, '"aaaa"') > <_sre.SRE_Match object; span=(0, 6), match='"aaaa"'> > >>>> re.match(string_re, '"aa\"aa"') > <_sre.SRE_Match object; span=(0, 4), match='"aa"'> > > How make the last match be the entire string '"aa\"aa"' ? > > cs You did not escape the string literals: >>> '"aa\"aa"' == '"aa"aa"' True Once you fix this for both the regex and the search string you get the expected match: >>> re.match('"(\\\\.|[^"])*"', '"aa\\"aa"') <_sre.SRE_Match object; span=(0, 8), match='"aa\\"aa"'> Raw strings make this a bit easier to read: >>> re.match(r'"(\\.|[^"])*"', r'"aa\"aa"') <_sre.SRE_Match object; span=(0, 8), match='"aa\\"aa"'> From applemask at gmail.com Thu Aug 9 13:33:58 2018 From: applemask at gmail.com (Apple) Date: Thu, 9 Aug 2018 10:33:58 -0700 (PDT) Subject: Embedded Python and multiprocessing on Windows? In-Reply-To: References: Message-ID: <36fe8bbe-753c-49a3-b3fc-5f9ce4b92f15@googlegroups.com> A quick update as I found a minor workaround, but I don't quite understand why it works this way or how I would fix it to work differently: So my program runs one script file, and multiprocessing commands from that script file seem to fail to spawn new processes. However, if that script file calls a function in a separate script file that it has imported, and that function calls multiprocessing functions, it all works exactly the way it should. So now I think my question is why does it behave this way, and is there anything I can do to fix the behavior in the first script? It's not the end of the world to need 2 scripts, but it sure would be a lot cleaner with just one. On Thursday, August 9, 2018 at 12:09:36 PM UTC-4, Apple wrote: > I've been working on a project involving embedding Python into a Windows application. I've got all of that working fine on the C++ side, but the script side seems to be hitting a dead end with multiprocessing. When my script tries to run the same multiprocessing code that works in a non-embedded environment, the code doesn't appear to be executed at all. > > A StackOverflow thread gave me a few ideas to try out, so I gave it a shot with these lines added: > > sys.argv = ['C:\\path\\to\\script.py'] > mp.set_executable("C:\\Python37\\python.exe") > p = mp.Process(target=somefunction) > p.start() > > Still no joy. However, a Python.exe window does pop up for a tenth of a second, so *something* is happening. > > So does multiprocessing actually work in embedded Python, or am I at a dead end here? My guess is that it's probably linked to Windows not having fork, so the combination of being embedded and having to spawn is making things go goofy. From cseberino at gmail.com Thu Aug 9 14:31:59 2018 From: cseberino at gmail.com (cseberino at gmail.com) Date: Thu, 9 Aug 2018 11:31:59 -0700 (PDT) Subject: Why this regex for string literals can't handle escaped quotes?.... '"(\\.|[^"])*"' In-Reply-To: References: <896892c5-aeff-4402-a7b2-1ba56261336d@googlegroups.com> Message-ID: Wow. Thanks. That cleared everything up. cs From python at mrabarnett.plus.com Thu Aug 9 15:05:01 2018 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 9 Aug 2018 20:05:01 +0100 Subject: name 'aLOCK' is not defined When I add aLOCK = threading.RLock() behind if __name__ == "__main__" In-Reply-To: <639b7d96.83da.1651f42d0e1.Coremail.xuanwu348@163.com> References: <639b7d96.83da.1651f42d0e1.Coremail.xuanwu348@163.com> Message-ID: On 2018-08-09 16:16, xuanwu348 wrote: > Hi team > > Good day > The problem I meet when I add "aLOCK = threading.RLock()" to PositionB, the program will report error "name 'aLOCK' is not defined ", > but when I change this code to PositionA, it can run normally, is there any difference for the code between 'if __name__ == "__main__:"', can you help me, thanks! > > The file I was attached, please change the extend ".pyx" to ".py", thanks, code as below, tried python2.7 and python3.4: > > import threading > import time > from multiprocessing import Process > > #PositionA > aLOCK = threading.RLock() > > def Save_runnedCMD(filename, strings): > aLOCK.acquire() > with open(filename, "at") as f: > f.write(str(strings) + "\n\r") > aLOCK.release() > > def runCMD(filename): > time.sleep(1) > cmd = "testtest" > Save_runnedCMD(filename, cmd) > > > def Thr_run(filename): > t = [] > for i in range(2): > tt = threading.Thread(target = runCMD, args=(filename,)) > tt.start() > t.append(tt) > for tt in t: > tt.join() > > if __name__ == "__main__": > filename = "./testaaaaa.log" > > #PositionB > #aLOCK = threading.RLock() > > while 1: > t1 = Process(target=Thr_run, args=(filename, )) > t1.start() > t1.join() > > Error info as below: > D:\Appppp>python testa.py > Exception in thread Thread-1: > Traceback (most recent call last): > File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner > self.run() > File "C:\Python34\lib\threading.py", line 869, in run > self._target(*self._args, **self._kwargs) > File "D:\Appppp\testa.py", line 15, in runCMD > Save_runnedCMD(filename, cmd) > File "D:\Appppp\testa.py", line 7, in Save_runnedCMD > aLOCK.acquire() > NameError: name 'aLOCK' is not defined > > Exception in thread Thread-2: > Traceback (most recent call last): > File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner > self.run() > File "C:\Python34\lib\threading.py", line 869, in run > self._target(*self._args, **self._kwargs) > File "D:\Appppp\testa.py", line 15, in runCMD > Save_runnedCMD(filename, cmd) > File "D:\Appppp\testa.py", line 7, in Save_runnedCMD > aLOCK.acquire() > NameError: name 'aLOCK' is not defined > When you run the script, __name__ will be "__main__". When script uses the multiprocessing module to run the function "Thr_run" in another process, it actually starts another instance of itself, and in that other instance, name _won't_ be "__main__". Here's a simple example: from multiprocessing import Process def test_func(): print('test_func') print('__name__ is {!a}'.format(__name__)) if __name__ == "__main__": p = Process(target=test_func) p.start() p.join() When run, it prints: __name__ is '__main__' __name__ is '__mp_main__' test_func So, if you put the lock at PositionB, it won't be created by the subprocess. From xuanwu348 at 163.com Thu Aug 9 19:59:46 2018 From: xuanwu348 at 163.com (jinliang) Date: Fri, 10 Aug 2018 07:59:46 +0800 Subject: =?UTF-8?Q?=E5=9B=9E=E5=A4=8D=EF=BC=9A_Re:_name_'aLOCK'_is_n?= =?UTF-8?Q?ot_defined_When_I_add_aLOCK?= =?UTF-8?Q?_=3D_threading.RLock()=0D__beh?= =?UTF-8?Q?ind_if_=5F=5Fname=5F=5F_=3D=3D_"=5F=5Fmain=5F=5F"?= Message-ID: Thanks, I think I get the answer. thanks all, have a good day! ????????? -------- ???? -------- ???? MRAB ??:2018-08-10 03:05 (GMT+08:00) ???? python-list at python.org ??? Re: name 'aLOCK' is not defined When I add aLOCK = threading.RLock() behind if __name__ == "__main__" On 2018-08-09 16:16, xuanwu348 wrote: > Hi team > > Good day > The problem I meet when I add "aLOCK = threading.RLock()" to PositionB, the program will report error "name 'aLOCK' is not defined ", > but when I change this code to PositionA, it can run normally, is there any difference for the code between 'if __name__ == "__main__:"', can you help me, thanks! > > The file I was attached, please change the extend ".pyx" to ".py", thanks, code as below, tried python2.7 and python3.4: > > import threading > import time > from multiprocessing import Process > > #PositionA > aLOCK = threading.RLock() > > def Save_runnedCMD(filename, strings): >????? aLOCK.acquire() >????? with open(filename, "at") as f: >????????? f.write(str(strings) + "\n\r") >????? aLOCK.release() > > def runCMD(filename): >????? time.sleep(1) >????? cmd = "testtest" >????? Save_runnedCMD(filename, cmd) > > > def Thr_run(filename): >????? t = [] >????? for i in range(2): >????????? tt = threading.Thread(target = runCMD, args=(filename,)) >????????? tt.start() >????????? t.append(tt) >????? for tt in t: >????????? tt.join() > > if __name__ == "__main__": >????? filename = "./testaaaaa.log" > >????? #PositionB >????? #aLOCK = threading.RLock() > >????? while 1: >????????? t1 = Process(target=Thr_run, args=(filename, )) >????????? t1.start() >????????? t1.join() > > Error info as below: > D:\Appppp>python testa.py > Exception in thread Thread-1: > Traceback (most recent call last): >??? File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner >????? self.run() >??? File "C:\Python34\lib\threading.py", line 869, in run >????? self._target(*self._args, **self._kwargs) >??? File "D:\Appppp\testa.py", line 15, in runCMD >????? Save_runnedCMD(filename, cmd) >??? File "D:\Appppp\testa.py", line 7, in Save_runnedCMD >????? aLOCK.acquire() > NameError: name 'aLOCK' is not defined > > Exception in thread Thread-2: > Traceback (most recent call last): >??? File "C:\Python34\lib\threading.py", line 921, in _bootstrap_inner >????? self.run() >??? File "C:\Python34\lib\threading.py", line 869, in run >????? self._target(*self._args, **self._kwargs) >??? File "D:\Appppp\testa.py", line 15, in runCMD >????? Save_runnedCMD(filename, cmd) >??? File "D:\Appppp\testa.py", line 7, in Save_runnedCMD >????? aLOCK.acquire() > NameError: name 'aLOCK' is not defined > When you run the script, __name__ will be "__main__". When script uses the multiprocessing module to run the function "Thr_run" in another process, it actually starts another instance of itself, and in that other instance, name _won't_ be "__main__". Here's a simple example: from multiprocessing import Process def test_func(): ???? print('test_func') print('__name__ is {!a}'.format(__name__)) if __name__ == "__main__": ???? p = Process(target=test_func) ???? p.start() ???? p.join() When run, it prints: __name__ is '__main__' __name__ is '__mp_main__' test_func So, if you put the lock at PositionB, it won't be created by the subprocess. -- https://mail.python.org/mailman/listinfo/python-list From 77minds at gmail.com Fri Aug 10 00:50:13 2018 From: 77minds at gmail.com (Karthikeyan A K) Date: Thu, 9 Aug 2018 21:50:13 -0700 (PDT) Subject: [Promotion] Book for Data Science Aspirants Message-ID: <0dd60651-7ae9-484d-8776-a38394f593a3@googlegroups.com> Hello Python people, Two years ago, my profession changed, I had to do data science work. When I started it, I (though I knew Python) found it to be extremely difficult and I had these doubts: What to learn? Where to learn? Am I doing things right? Where can I find a person who would guide me in the right path? Were the questions that arose in my mind. I had no proper guide and it was pain. So I wrote this book "How To Become A Data Scientist" https://www.amazon.com/dp/B07FYVTNX7 , a guide for those who would like to become Data Scientist. I have crystallized two years of my struggle and answers to it in this book. I hope this book would help you. - Karthikeyan A K From Karsten.Hilbert at gmx.net Fri Aug 10 02:15:09 2018 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Fri, 10 Aug 2018 08:15:09 +0200 Subject: name 'aLOCK' is not defined When I add aLOCK = threading.RLock() behind if __name__ == "__main__" In-Reply-To: <231b0eff.8721.1651f80e3a8.Coremail.xuanwu348@163.com> References: <639b7d96.83da.1651f42d0e1.Coremail.xuanwu348@163.com> <20180809155145.GG3862@hermes.hilbert.loc> <231b0eff.8721.1651f80e3a8.Coremail.xuanwu348@163.com> Message-ID: <20180810061509.GA4304@hermes.hilbert.loc> On Fri, Aug 10, 2018 at 12:24:25AM +0800, xuanwu348 wrote: > Yes, move the code from positionA(can run normally) to positionB(exception with name undefined) > I find this content "https://docs.python.org/3.3/tutorial/classes.html#python-scopes-and-namespaces" > But I still don't undewrstand the differenct of scopes-and-namespaces between positionA and positionB, > > I think the variable "aLock" at these positionA or positionB are all global. When something goes wrong in an unexpected way: test your assumptions ;-) Best, Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B From leo at superlel.me Fri Aug 10 02:28:27 2018 From: leo at superlel.me (=?UTF-8?Q?L=c3=a9o_El_Amri?=) Date: Fri, 10 Aug 2018 08:28:27 +0200 Subject: Embedded Python and multiprocessing on Windows? In-Reply-To: <36fe8bbe-753c-49a3-b3fc-5f9ce4b92f15@googlegroups.com> References: <36fe8bbe-753c-49a3-b3fc-5f9ce4b92f15@googlegroups.com> Message-ID: <4af5eb36-8c4a-2126-f786-c199e59d3878@superlel.me> On 09/08/2018 19:33, Apple wrote:> So my program runs one script file, and multiprocessing commands from that script file seem to fail to spawn new processes. > > However, if that script file calls a function in a separate script file that it has imported, and that function calls multiprocessing functions, it all works exactly the way it should. > On Thursday, August 9, 2018 at 12:09:36 PM UTC-4, Apple wrote: >> I've been working on a project involving embedding Python into a Windows application. I've got all of that working fine on the C++ side, but the script side seems to be hitting a dead end with multiprocessing. When my script tries to run the same multiprocessing code that works in a non-embedded environment, the code doesn't appear to be executed at all. >> >> Still no joy. However, a Python.exe window does pop up for a tenth of a second, so *something* is happening. That may be something simple: Did you actually protected the entry-point of your Python script with if __name__ == '__main__': ? From steve+comp.lang.python at pearwood.info Fri Aug 10 02:51:44 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 10 Aug 2018 06:51:44 +0000 (UTC) Subject: name 'aLOCK' is not defined When I add aLOCK = threading.RLock() behind if __name__ == "__main__" References: <639b7d96.83da.1651f42d0e1.Coremail.xuanwu348@163.com> <20180809155145.GG3862@hermes.hilbert.loc> <231b0eff.8721.1651f80e3a8.Coremail.xuanwu348@163.com> <20180810061509.GA4304@hermes.hilbert.loc> Message-ID: On Fri, 10 Aug 2018 08:15:09 +0200, Karsten Hilbert wrote: > On Fri, Aug 10, 2018 at 12:24:25AM +0800, xuanwu348 wrote: > >> Yes, move the code from positionA(can run normally) to >> positionB(exception with name undefined) I find this content >> "https://docs.python.org/3.3/tutorial/classes.html#python-scopes-and- namespaces" >> But I still don't undewrstand the differenct of scopes-and-namespaces >> between positionA and positionB, >> >> I think the variable "aLock" at these positionA or positionB are all >> global. > > When something goes wrong in an unexpected way: test your assumptions > ;-) xuanwu348's assumptions are correct. aLock is a global, in both positions. The problem is not the scope of the variable, but whether or not the variable is assigned to or not. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From rvadiga at gmail.com Fri Aug 10 07:43:15 2018 From: rvadiga at gmail.com (Venkatesh Adiga) Date: Fri, 10 Aug 2018 17:13:15 +0530 Subject: Import issue in python packages Message-ID: Hi All, I am facing issue with python package import. In my project, I have many directories with different python classes defined within them. I am trying to import those classes in another python program, but not able to import them. Here are my directories and file structures. src/platform/operatingsys.py : Defined OperatingSystem Class src/platform/ipc.py : Defined IPC class src/platform/MyThread.py: Defined MyThread class src/platform/__init__.py - zero sized file src/utils/abc.py src/utils/def.py src/utils/__init__.py src/api/xyz.py src/api/__init__.py src/unittest/TestSuite.py - Defined UnitTest Class src/unittest/__init__.py src/__init__.py Now, I am trying to use the above classes in another python program( TestPackage.py) as below: *from src.platform.operatingsys import OperatingSystem* ..... -------------- Execution of TestPackage.py throws an error as below ImportError: No module named 'src' Currently I have a working code which prefixes sys.path with every directory defined in the package list before import any classes. sys.path.append("src/unittest/") import OperatingSystem But I do not have the hard-coded path and append to sys.path variable. So my question is: 1) Is there any better way of having the sys.path variable appended by directory listing? 2) What changes i need to do so that my import statement looks like below: *from src.platform.operatingsys import OperatingSystem* Please suggest... Thanks Venkat From bgailer at gmail.com Fri Aug 10 09:13:05 2018 From: bgailer at gmail.com (Bob Gailer) Date: Fri, 10 Aug 2018 09:13:05 -0400 Subject: Python MySQL Guide In-Reply-To: <0fbeeedc-178d-40c1-abce-5be84e7aa66d@googlegroups.com> References: <0fbeeedc-178d-40c1-abce-5be84e7aa66d@googlegroups.com> Message-ID: Thank you for this offer. My reaction is I don't like having to scroll through one very long page to find what I'm looking for. Might you consider breaking it up into a number of smaller pages and giving an index as the main page? On Aug 9, 2018 5:18 PM, wrote: > Refer this complete guide on working with Python and MySQL > > https://pynative.com/python-mysql-tutorial/ > -- > https://mail.python.org/mailman/listinfo/python-list > From aenkaa at gmail.com Fri Aug 10 09:35:27 2018 From: aenkaa at gmail.com (aenkaa at gmail.com) Date: Fri, 10 Aug 2018 06:35:27 -0700 (PDT) Subject: Multiprocessing "Pool" aborts without any error message or return code? (Python 3.6.4, cygwin 32 bit, Windows Server 2012) Message-ID: Please refer to: https://docs.python.org/3.6/library/multiprocessing.html the first example program. If I run it on Windows 10 (Python 3.6.4) or Linux Mint 18 (Python 3.5.2), it works as expected. On Windows Server 2012, however, I get no (none!) output, no matter if I run with admin rights or not: adminnkj at DTDKCPHAS1060 ~ $ python3 -V Python 3.6.4 adminnkj at DTDKCPHAS1060 ~ $ cat test.py from multiprocessing import Pool def f(x): return x*x if __name__ == '__main__': p = Pool(5) print(p.map(f, [1, 2, 3])) adminnkj at DTDKCPHAS1060 ~ $ python3 test.py ---> NB: no output <--- adminnkj at DTDKCPHAS1060 ~ $ echo $? 0 ---> NB: Normal exit code <--- I am a bit lost, running with -v -vv or -vvv does not help me much (here are the last lines of the latter): (cut) import 'multiprocessing.connection' # <_frozen_importlib_external.SourceFileLoader object at 0xffc913f0> import 'multiprocessing.queues' # <_frozen_importlib_external.SourceFileLoader object at 0xffc7ab10> # trying /usr/lib/python3.6/multiprocessing/synchronize.cpython-36m-i386-cygwin.dll # trying /usr/lib/python3.6/multiprocessing/synchronize.abi3.dll # trying /usr/lib/python3.6/multiprocessing/synchronize.dll # trying /usr/lib/python3.6/multiprocessing/synchronize.py # /usr/lib/python3.6/multiprocessing/__pycache__/synchronize.cpython-36.pyc matches /usr/lib/python3.6/multiprocessing/synchronize.py # code object from '/usr/lib/python3.6/multiprocessing/__pycache__/synchronize.cpython-36.pyc' import 'multiprocessing.synchronize' # <_frozen_importlib_external.SourceFileLoader object at 0xffc91af0> # trying /usr/lib/python3.6/multiprocessing/popen_fork.cpython-36m-i386-cygwin.dll # trying /usr/lib/python3.6/multiprocessing/popen_fork.abi3.dll # trying /usr/lib/python3.6/multiprocessing/popen_fork.dll # trying /usr/lib/python3.6/multiprocessing/popen_fork.py # /usr/lib/python3.6/multiprocessing/__pycache__/popen_fork.cpython-36.pyc matches /usr/lib/python3.6/multiprocessing/popen_fork.py # code object from '/usr/lib/python3.6/multiprocessing/__pycache__/popen_fork.cpython-36.pyc' import 'multiprocessing.popen_fork' # <_frozen_importlib_external.SourceFileLoader object at 0xffc57190> then it exits with no further output. Best regards, Niels Kristian Jensen (from Denmark) From vishalhule24 at gmail.com Fri Aug 10 12:39:40 2018 From: vishalhule24 at gmail.com (Vishal Hule) Date: Fri, 10 Aug 2018 22:09:40 +0530 Subject: Python MySQL Guide In-Reply-To: References: <0fbeeedc-178d-40c1-abce-5be84e7aa66d@googlegroups.com> Message-ID: Sure, no problem. I also added a table of contents at the start of the article you can also refer that On Fri, 10 Aug 2018, 6:43 pm Bob Gailer, wrote: > Thank you for this offer. My reaction is I don't like having to scroll > through one very long page to find what I'm looking for. Might you consider > breaking it up into a number of smaller pages and giving an index as the > main page? > > On Aug 9, 2018 5:18 PM, wrote: > >> Refer this complete guide on working with Python and MySQL >> >> https://pynative.com/python-mysql-tutorial/ >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > From __peter__ at web.de Fri Aug 10 13:23:22 2018 From: __peter__ at web.de (Peter Otten) Date: Fri, 10 Aug 2018 19:23:22 +0200 Subject: Import issue in python packages References: Message-ID: Venkatesh Adiga wrote: > Hi All, > > I am facing issue with python package import. > In my project, I have many directories with different python classes > defined within them. I am trying to import those classes in another python > program, but not able to import them. > Here are my directories and file structures. > src/platform/operatingsys.py : Defined OperatingSystem Class > src/platform/ipc.py : Defined IPC class > src/platform/MyThread.py: Defined MyThread class > src/platform/__init__.py - zero sized file > > src/utils/abc.py > src/utils/def.py > src/utils/__init__.py > > src/api/xyz.py > src/api/__init__.py > > src/unittest/TestSuite.py - Defined UnitTest Class > src/unittest/__init__.py > src/__init__.py > > > Now, I am trying to use the above classes in another python program( > TestPackage.py) as below: > > > *from src.platform.operatingsys import OperatingSystem* > ..... > -------------- > Execution of TestPackage.py throws an error as below > > ImportError: No module named 'src' > > Currently I have a working code which prefixes sys.path with every > directory defined in the package list before import any classes. > > sys.path.append("src/unittest/") > import OperatingSystem > > But I do not have the hard-coded path and append to sys.path variable. > So my question is: > 1) Is there any better way of having the sys.path variable appended by > directory listing? > 2) What changes i need to do so that my import statement looks like > below: > *from src.platform.operatingsys import OperatingSystem* > > Please suggest... Ensure that src is in your PYTHONPATH, preferrably as an absolute path. Then import the packages and modules without the src prefix, e. g. sys.path.append("/path/to/src") import platform.operatingsys If you insist on src as part of the import the directory's parent needs to be in the path: sys.path.append("/path/to") import src.platform.operatingsys Note that it's better to avoid manipulating sys.path from within your scripts. If at all possible install your packages or at least add .pth files or a PYTHONPATH environment variable. From sharan.basappa at gmail.com Fri Aug 10 13:46:06 2018 From: sharan.basappa at gmail.com (Sharan Basappa) Date: Fri, 10 Aug 2018 10:46:06 -0700 (PDT) Subject: plot not showing up Message-ID: <1bbf9124-5659-43d1-892c-d38c20505595@googlegroups.com> What is wrong with the following code. Python does not show the plot. from sklearn.datasets import load_digits from sklearn.cluster import KMeans import matplotlib.pyplot as plt digits = load_digits() digits.data.shape kmeans = KMeans(n_clusters=10,random_state=0) clusters = kmeans.fit_predict(digits.data) kmeans.cluster_centers_.shape fig, ax = plt.subplots(2,5,figsize=(8,3)) centers = kmeans.cluster_centers_.reshape(10,8,8) for axi, center in zip(ax.flat, centers): axi.set(xticks=[], yticks=[]) axi.imshow(center, interpolation='nearest', cmap=plt.cm.binary) From arj.python at gmail.com Fri Aug 10 13:51:03 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Fri, 10 Aug 2018 21:51:03 +0400 Subject: plot not showing up In-Reply-To: <1bbf9124-5659-43d1-892c-d38c20505595@googlegroups.com> References: <1bbf9124-5659-43d1-892c-d38c20505595@googlegroups.com> Message-ID: try adding plt.show() Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius On Fri, 10 Aug 2018, 21:50 Sharan Basappa, wrote: > What is wrong with the following code. Python does not show the plot. > > from sklearn.datasets import load_digits > from sklearn.cluster import KMeans > import matplotlib.pyplot as plt > > digits = load_digits() > digits.data.shape > > kmeans = KMeans(n_clusters=10,random_state=0) > clusters = kmeans.fit_predict(digits.data) > kmeans.cluster_centers_.shape > > fig, ax = plt.subplots(2,5,figsize=(8,3)) > centers = kmeans.cluster_centers_.reshape(10,8,8) > for axi, center in zip(ax.flat, centers): > axi.set(xticks=[], yticks=[]) > axi.imshow(center, interpolation='nearest', cmap=plt.cm.binary) > -- > https://mail.python.org/mailman/listinfo/python-list > From applemask at gmail.com Fri Aug 10 15:06:39 2018 From: applemask at gmail.com (applemask at gmail.com) Date: Fri, 10 Aug 2018 12:06:39 -0700 (PDT) Subject: Embedded Python and multiprocessing on Windows? In-Reply-To: References: <36fe8bbe-753c-49a3-b3fc-5f9ce4b92f15@googlegroups.com> <4af5eb36-8c4a-2126-f786-c199e59d3878@superlel.me> Message-ID: <7696e13b-c8f1-44ab-b75e-24067720c4a5@googlegroups.com> On Friday, August 10, 2018 at 2:28:45 AM UTC-4, L?o El Amri wrote: > That may be something simple: Did you actually protected the entry-point > of your Python script with if __name__ == '__main__': ? That was my first thought too; the script technically doesn't have top-level code, so I figured I didn't need a main catch. Just to be sure, I added one and it still didn't resolve. The function in the second script that correctly multiprocesses doesn't have a main (or top level code) either. I'm wondering if it's related to how I call the script in code. I'm using boost.python, but I'm not sure I'm doing it entirely correctly (the boost.python documentation is weirdly hazy on the semantics of calling script files, but is drowning in examples of how to execute hard-coded lines, which seems insane to me; who on earth would go to the trouble to embed an interpreter to run hardcoded Python strings?) Anyway, those lines look like this: py::object main_module = py::import("__main__"); py::dict global = py::extract(main_module.attr("__dict__")); py::object result = py::exec_file(scriptName, global, global); py::object runner = global["EntryFunction"]; retval = py::extract(runner(py::ptr(taskPtr))); // we pass a pointer to the entry function From inhahe at gmail.com Fri Aug 10 18:40:11 2018 From: inhahe at gmail.com (inhahe) Date: Fri, 10 Aug 2018 18:40:11 -0400 Subject: Can't figure out how to do something using ctypes (and maybe struct?) Message-ID: I need to make a list of instances of a Structure, then I need to make an instance of another Structure, one of the fields of which needs to be an arbitrary-length array of pointers to the instances in the list. How do I do that? Just in case it helps, I'll include what I tried that didn't work: ------ class NoteEvent(ctypes.Structure): _fields_ = [('type', ctypes.c_int), ('byteSize', ctypes.c_int), ('deltaFrames', ctypes.c_int), ('flags', ctypes.c_int), ('noteLength', ctypes.c_int), ('noteOffset', ctypes.c_int), ('commandCode', ctypes.c_char), ('noteNumber', ctypes.c_char), ('velocity', ctypes.c_char)] def mkVstEvents(events): class Events(ctypes.Structure): _fields_ = [('numEvents', ctypes.c_int), ('reserved', ctypes.c_int), ('eventspointerarray', ctypes.c_void_p * len(events))] return Events(len(events), 0, tuple([ctypes.pointer(event) for event in events])) def mkNoteEvent(deltaFrames, flags, noteNumber, channel, velocity, noteLength, noteOffset, detune, noteOffVelocity): vstEvent = NoteEvent(type=1, bytesize=11, deltaFrames=deltaFrames, flags=1, noteLength=0, noteOffset=0, commandCode=chr(144+channel), noteNumber=chr(noteNumber), velocity=chr(velocity)) return vstEvent def onSendMidiButton(self, event): plugins[0].plugin.process_events(ctypes.pointer(mkVstEvents([mkNoteEvent(deltaFrames=0, flags=1, noteNumber=60, channel=0, velocity=127, noteLength=0, noteOffset=0, detune=0, noteOffVelocity=0)]))) ------ Here's the error I got: ------ Traceback (most recent call last): File "soundshop.9.1.2.py", line 216, in onSendMidiButton noteOffVelocity=0)]))) File "soundshop.9.1.2.py", line 76, in mkVstEvents return Events(len(events), 0, tuple([ctypes.pointer(event) for event in events])) RuntimeError: (c_void_p_Array_1) : incompatible types, LP_NoteEvent instance instead of c_void_p instance ------ I also tried to use struct.pack: ------ def mkVstEvents(events): st = struct.pack("=ii"+"L"*len(events), len(events), 0, *(ctypes.pointer(event) for event in events)) return st ------- Here's the error I got: ------ Traceback (most recent call last): File "soundshop.9.3.py", line 213, in onSendMidiButton noteOffVelocity=0)]))) File "soundshop.9.3.py", line 72, in mkVstEvents st = struct.pack("=ii"+"p"*len(events), len(events), 0, *(ctypes.pointer(event) for event in events)) struct.error: argument for 'p' must be a string ------ I also tried replacing "p"*len(events) with "i"*len(events) and also with "L"*len(events), both of which gave me errors. Thanks for your help. From steve+comp.lang.python at pearwood.info Fri Aug 10 20:15:43 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 11 Aug 2018 00:15:43 +0000 (UTC) Subject: Can't figure out how to do something using ctypes (and maybe struct?) References: Message-ID: On Fri, 10 Aug 2018 18:40:11 -0400, inhahe wrote: > I need to make a list of instances of a Structure, then I need to make > an instance of another Structure, one of the fields of which needs to be > an arbitrary-length array of pointers to the instances in the list. How > do I do that? > > Just in case it helps, I'll include what I tried that didn't work: How about simplifying your example to the smallest and simplest example of the problem? Your example has: - two functions; - one method that seems to have become unattached from its class; - two classes; - using 12 different fields. Surely not all of that detail is specific to the problem you are happening. If you can simplify the problem, the solution may be more obvious. It might help to read this: http://sscce.org/ By the way, unrelated to your specific problem but possibly relevant elsewhere, you have this function: > def mkVstEvents(events): > class Events(ctypes.Structure): > _fields_ = [ ... ] > return Events( ... ) You might not be aware of this, but that means that every time you call mkVstEvents, you get a singleton instance of a new and distinct class that just happens to have the same name and layout. So if you did this: a = mkVstEvents( ... ) b = mkVstEvents( ... ) then a and b would *not* be instances of the same class: isinstance(a, type(b)) # returns False isinstance(b, type(a)) # returns False type(a) == type(b) # also False Each time you call the function, it creates a brand new class, always called Events, creates a single instance of that class, and returns it. That is especially wasteful of memory, since classes aren't small. py> class Events(ctypes.Structure): ... pass ... py> sys.getsizeof(Events) 508 Unless that's what you intended, you ought to move the class outside of the function. class Events(ctypes.Structure): _fields_ = [ ... ] def mkVstEvents(events): return Events( ... ) -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From rvadiga at gmail.com Sat Aug 11 04:46:56 2018 From: rvadiga at gmail.com (Venkatesh Adiga) Date: Sat, 11 Aug 2018 14:16:56 +0530 Subject: Import issue in python packages In-Reply-To: References: Message-ID: Thanks Peter... What are the ways to update outside of the python program without defining environment variable for source code path.... Otherwise can I do it once in during initialization of sys.path update? On Fri, 10 Aug 2018, 10:56 pm Peter Otten, <__peter__ at web.de> wrote: > Venkatesh Adiga wrote: > > > Hi All, > > > > I am facing issue with python package import. > > In my project, I have many directories with different python classes > > defined within them. I am trying to import those classes in another > python > > program, but not able to import them. > > Here are my directories and file structures. > > src/platform/operatingsys.py : Defined OperatingSystem Class > > src/platform/ipc.py : Defined IPC class > > src/platform/MyThread.py: Defined MyThread class > > src/platform/__init__.py - zero sized file > > > > src/utils/abc.py > > src/utils/def.py > > src/utils/__init__.py > > > > src/api/xyz.py > > src/api/__init__.py > > > > src/unittest/TestSuite.py - Defined UnitTest Class > > src/unittest/__init__.py > > src/__init__.py > > > > > > Now, I am trying to use the above classes in another python program( > > TestPackage.py) as below: > > > > > > *from src.platform.operatingsys import OperatingSystem* > > ..... > > -------------- > > Execution of TestPackage.py throws an error as below > > > > ImportError: No module named 'src' > > > > Currently I have a working code which prefixes sys.path with every > > directory defined in the package list before import any classes. > > > > sys.path.append("src/unittest/") > > import OperatingSystem > > > > But I do not have the hard-coded path and append to sys.path variable. > > So my question is: > > 1) Is there any better way of having the sys.path variable appended by > > directory listing? > > 2) What changes i need to do so that my import statement looks like > > below: > > *from src.platform.operatingsys import OperatingSystem* > > > > Please suggest... > > Ensure that src is in your PYTHONPATH, preferrably as an absolute path. > Then import the packages and modules without the src prefix, e. g. > > sys.path.append("/path/to/src") > import platform.operatingsys > > If you insist on src as part of the import the directory's parent needs to > be in the path: > > sys.path.append("/path/to") > import src.platform.operatingsys > > Note that it's better to avoid manipulating sys.path from within your > scripts. If at all possible install your packages or at least add .pth > files > or a PYTHONPATH environment variable. > > -- > https://mail.python.org/mailman/listinfo/python-list > From __peter__ at web.de Sat Aug 11 04:54:54 2018 From: __peter__ at web.de (Peter Otten) Date: Sat, 11 Aug 2018 10:54:54 +0200 Subject: Can't figure out how to do something using ctypes (and maybe struct?) References: Message-ID: inhahe wrote: > I need to make a list of instances of a Structure, then I need to make an > instance of another Structure, one of the fields of which needs to be an > arbitrary-length array of pointers to the instances in the list. How do I > do that? > > Just in case it helps, I'll include what I tried that didn't work: > ------ > class NoteEvent(ctypes.Structure): > _fields_ = [('type', ctypes.c_int), > ('byteSize', ctypes.c_int), > ('deltaFrames', ctypes.c_int), > ('flags', ctypes.c_int), > ('noteLength', ctypes.c_int), > ('noteOffset', ctypes.c_int), > ('commandCode', ctypes.c_char), > ('noteNumber', ctypes.c_char), > ('velocity', ctypes.c_char)] > > def mkVstEvents(events): > class Events(ctypes.Structure): > _fields_ = [('numEvents', ctypes.c_int), > ('reserved', ctypes.c_int), > ('eventspointerarray', ctypes.c_void_p * len(events))] > return Events(len(events), 0, tuple([ctypes.pointer(event) for event in > events])) ... > RuntimeError: (c_void_p_Array_1) : > incompatible types, LP_NoteEvent instance instead of c_void_p instance Two things that you can try: > ('eventspointerarray', ctypes.c_void_p * len(events))] Either change c_void_p to POINTER(NoteEvent) like https://stackoverflow.com/questions/7015487/ctypes-variable-length-structures or cast the NoteEvent pointer to void *: > return Events(len(events), 0, tuple([ctypes.pointer(event) for event in > events])) ... cast(pointer(event), c_void_p) ... From __peter__ at web.de Sat Aug 11 05:20:05 2018 From: __peter__ at web.de (Peter Otten) Date: Sat, 11 Aug 2018 11:20:05 +0200 Subject: Import issue in python packages References: Message-ID: Venkatesh Adiga wrote: > Thanks Peter... What are the ways to update outside of the python program > without defining environment variable for source code path.... Otherwise > can I do it once in during initialization of sys.path update? I don't understand the question. Your options are - install your modules/packages in a directory already in sys.path - put .pth file in a directory already in sys.path - set PYTHONPATH - manipulate sys.path at runtime In my experience the last option has a tendency to break things which is why I prefer any of the other ones. It's your choice however. From jonas at wielicki.name Sat Aug 11 10:29:45 2018 From: jonas at wielicki.name (Jonas Wielicki) Date: Sat, 11 Aug 2018 16:29:45 +0200 Subject: [ANN] aioxmpp 0.10.1 released Message-ID: <6022808.m6R7XdkQ7X@sinistra> Dear subscribers, We are pleased to announce the release of aioxmpp 0.10.1. The current release can be obtained from GitHub [1] (check out the v0.10.1 tag) or PyPI [2]. The HTML documentation can be found at [3]. Examples can be found in the GitHub repository, in the examples sub directory. aioxmpp is a Python library based on asyncio. It implements the client side of the XMPP protocol (RFC 6120 and others). For a more detailed description of the package, please review the README of the package on GitHub [1] or on PyPI [2]. For more information on XMPP, please see [8]. aioxmpp is licensed under the terms of the GNU Lesser General Public License Version 3.0 or later. Version 0.10.1 is a bugfix release with possible security implications. It was discovered that two types of specifically crafted (invalid) IQ stanzas could make aioxmpp 0.10.0 disconnect, leading to a Denial-of-Service type effect. We decided to push a release quickly to fix this issue. Thanks to Martin for discovering and reporting the issue. Bugs, feature requests, patches and questions can be directed to either the aioxmpp mailing list [4], the GitHub issue tracker [5] or the XMPP Multi-User chat [6], whatever floats your boat. Please direct security-relevant issue reports directly to me (jonas at wielicki.name), preferably encrypted using my GPG public key [7]. kind regards, Jonas [1]: https://github.com/horazont/aioxmpp [2]: https://pypi.python.org/pypi/aioxmpp [3]: https://docs.zombofant.net/aioxmpp/0.10/ [4]: https://lists.zombofant.net/mailman/listinfo/aioxmpp-devel [5]: https://github.com/horazont/aioxmpp/issues [6]: aioxmpp at conference.zombofant.net [7]: https://sks-keyservers.net/pks/lookup?op=get&search=0xE5EDE5AC679E300F AA5A 78FF 508D 8CF4 F355 F682 E5ED E5AC 679E 300F [8]: https://xmpp.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part. URL: From wfgazdzik at gmail.com Sat Aug 11 16:01:44 2018 From: wfgazdzik at gmail.com (wfgazdzik at gmail.com) Date: Sat, 11 Aug 2018 13:01:44 -0700 (PDT) Subject: Python-Tkinter issue. Multiple overlaping event routines called by single click Message-ID: I have a main window open. Then I open a tk.TopLevel dialog window giving the user multiple choices. He selects one, the corresponding event is executed. Then in the underlining main window, just by chance there is another button exactly under the mouse click in the TopLevel dialog window. Its corresponding event is then triggered. How can I keep the main window button that just happens to be in the wrong place from being triggered? From __peter__ at web.de Sat Aug 11 17:58:42 2018 From: __peter__ at web.de (Peter Otten) Date: Sat, 11 Aug 2018 23:58:42 +0200 Subject: Python-Tkinter issue. Multiple overlaping event routines called by single click References: Message-ID: wfgazdzik at gmail.com wrote: > I have a main window open. Then I open a tk.TopLevel dialog window giving > the user multiple choices. He selects one, the corresponding event is > executed. Then in the underlining main window, just by chance there is > another button exactly under the mouse click in the TopLevel dialog > window. Its corresponding event is then triggered. > > How can I keep the main window button that just happens to be in the wrong > place from being triggered? Please post a small script that shows the behaviour described above. That will give us some common ground to talk about. From python at mrabarnett.plus.com Sat Aug 11 20:30:43 2018 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 12 Aug 2018 01:30:43 +0100 Subject: Python-Tkinter issue. Multiple overlaping event routines called by single click In-Reply-To: References: Message-ID: <61cf1d16-d612-0c07-6369-a9bc07106265@mrabarnett.plus.com> On 2018-08-11 21:01, wfgazdzik at gmail.com wrote: > I have a main window open. Then I open a tk.TopLevel dialog window giving the user multiple choices. He selects one, the corresponding event is executed. Then in the underlining main window, just by chance there is another button exactly under the mouse click in the TopLevel dialog window. Its corresponding event is then triggered. > > How can I keep the main window button that just happens to be in the wrong place from being triggered? > The handler should return the string "break" to prevent the event from propagating further. Are you doing that? It's surprising how far you can go without it before running into a problem! From steve+comp.lang.python at pearwood.info Sat Aug 11 20:41:24 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 12 Aug 2018 00:41:24 +0000 (UTC) Subject: Python-Tkinter issue. Multiple overlaping event routines called by single click References: Message-ID: On Sat, 11 Aug 2018 13:01:44 -0700, wfgazdzik wrote: > I have a main window open. Then I open a tk.TopLevel dialog window > giving the user multiple choices. He selects one, the corresponding > event is executed. Then in the underlining main window, just by chance > there is another button exactly under the mouse click in the TopLevel > dialog window. Its corresponding event is then triggered. Sounds to me that the user is clicking twice, once in the dialog, and then a second time just as it disappears and the main window takes focus. Possibly they are trying to double-click. Or their mouse is faulty. Unless you can replicate this with multiple users, the most likely cause is user-error. And unless you can eliminate user-error, trying to work- around users who click randomly is a nightmare. How do you decide which clicks are intended and which are not? > How can I keep the main window button that just happens to be in the > wrong place from being triggered? If you put in a delay between enacting the event and closing the dialog, I reckon the problem will go away... but instead you'll have two click events in the dialog. But it seems like an interesting experiment... put time.sleep(0.3) at the end of the event handler and see what happens. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From steve+comp.lang.python at pearwood.info Sat Aug 11 21:12:41 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 12 Aug 2018 01:12:41 +0000 (UTC) Subject: Python-Tkinter issue. Multiple overlaping event routines called by single click References: <61cf1d16-d612-0c07-6369-a9bc07106265@mrabarnett.plus.com> Message-ID: On Sun, 12 Aug 2018 01:30:43 +0100, MRAB wrote: > On 2018-08-11 21:01, wfgazdzik at gmail.com wrote: >> I have a main window open. Then I open a tk.TopLevel dialog window >> giving the user multiple choices. He selects one, the corresponding >> event is executed. Then in the underlining main window, just by chance >> there is another button exactly under the mouse click in the TopLevel >> dialog window. Its corresponding event is then triggered. >> >> How can I keep the main window button that just happens to be in the >> wrong place from being triggered? >> > The handler should return the string "break" to prevent the event from > propagating further. Are you doing that? It's surprising how far you can > go without it before running into a problem! I think you are mistaken: https://stackoverflow.com/a/12357536 but since the description of the problem is so vague, it is hard to tell exactly what's happening. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From gzaidman at redhat.com Sun Aug 12 11:42:17 2018 From: gzaidman at redhat.com (Gal Zaidman) Date: Sun, 12 Aug 2018 18:42:17 +0300 Subject: Packaging for python2/3 compatibility Message-ID: Hello all, I'm a member of the RHEV integration team and for the past few weeks I've been working on packaging Ovirt for python3 compatibility for Fedora, and we have very clear guidelines for packaging python packages for Fedora: https://fedoraproject.org/wiki/Packaging:Python#Byte_compiling and that is basically a package name starts with python2/3-*package-name*. But in RHEL or Centos, we have some packages named: python-*package-name*, *package-name*-python and so on, one example is libselinux, which in Fedora is: python2/3-libselinux and on Centos\RHEL is libselinux-python When I re-write the spec file I want to avoid (as much as possible) from if/else like if rhel require and if fedora require, I want to know if there is a correct way of doing so without using if/else everywhere in the spec file? BTW I thought of creating a macro function that gets the package name and an argument for the location of python for rhel and returns the correct python package name, and if that is the best way, I had trouble in finding examples of spec macro functions with conditions within them so I would love to get a good example of such macro function. Thanks, Gal From leo at superlel.me Sun Aug 12 13:33:49 2018 From: leo at superlel.me (=?UTF-8?Q?L=c3=a9o_El_Amri?=) Date: Sun, 12 Aug 2018 19:33:49 +0200 Subject: asyncio: Warning message when waiting for an Event set by AbstractLoop.add_reader In-Reply-To: <09d9c579-7d03-f740-1b40-48535afdeafa@superlel.me> References: <09d9c579-7d03-f740-1b40-48535afdeafa@superlel.me> Message-ID: I found out what was the problem. The behavior of my "reader" (The callback passed to AbstractEventLoop.add_reader()) is to set an event. This event is awaited for in a coroutine which actually reads what is written on a pipe. The execution flow is the following: * NEW LOOP TURN * The selector awaits for read-ready state on a file descriptor AND The coroutine awaits for the event * At some point, the file descriptor becomes read-ready, the selector queue the callback to set the event on the loop * The callback put on the queue is called and the event is set * NEW LOOP TURN * The selector awaits for read-ready state on a file descriptor AND The coroutine awaits for the event * The file descriptor is read-ready, so the selector queue the callback to set the event on the loop * The corouting is resumed, because the event is now set, the corouting then reads what was written on the file descriptor, then clears the event and awaits again for it * The callback put on the queue is called and the event is set * NEW LOOP TURN * The selector awaits for read-ready state on a file descriptor AND The coroutine awaits for the event * The corouting is resumed, because the event was set at the end of the last turn, so the coroutine attempts a read, but blocks on it, because the file descriptor is actually not read-ready Rince and repeat. This behavior actually depends on how the loop is implemented. So the best practice here is to actually do the read from the callback you gives to AbstractEventLoop.add_reader(). I think we should put a mention about it in the documentation. From aenkaa at gmail.com Mon Aug 13 03:33:22 2018 From: aenkaa at gmail.com (Niels Kristian Jensen) Date: Mon, 13 Aug 2018 00:33:22 -0700 (PDT) Subject: Multiprocessing "Pool" aborts without any error message or return code? (Python 3.6.4, cygwin 32 bit, Windows Server 2012) In-Reply-To: References: Message-ID: <709a2613-65f2-4095-ba05-f45b46a2615d@googlegroups.com> fredag den 10. august 2018 kl. 15.35.46 UTC+2 skrev Niels Kristian Jensen: > Please refer to: > (cut) It appears, that Python is simply not supported on Cygwin (!): https://bugs.python.org/issue30563 Best regards, Niels Kristian From redstone-cold at 163.com Mon Aug 13 07:51:17 2018 From: redstone-cold at 163.com (iMath) Date: Mon, 13 Aug 2018 04:51:17 -0700 (PDT) Subject: right way to use zipimport, zipimport.ZipImportError: not a Zip file Message-ID: <7aa781a2-a404-4d3c-862a-1de8afe211ba@googlegroups.com> The same question also posted here https://stackoverflow.com/questions/51820473/right-way-to-use-zipimport-zipimport-zipimporterror-not-a-zip-file I have managed to use [this module][1] without installation - just import it from path to use , import sys url = 'https://example.com' sys.path.insert(0, r'C:\Users\i\Downloads\you-get-0.4.1128\src') # from you_get import common common.any_download(url, info_only=True)#NoneType It seems possible in Python to use `zipimport` to directly use the zip archive of the module without extraction, I wonder what is the right way to use `zipimport`, a simple trying like the following just gives the exception . I downloaded the file from [here][2] , the file `C:\Users\i\Downloads\you-get-0.4.1128.zip` does exist and isn't corrupted. >>> import zipimport >>> zipimport.zipimporter(r'C:\Users\i\Downloads\you-get-0.4.1128.zip') Traceback (most recent call last): File "", line 1, in zipimport.zipimporter(r'C:\Users\i\Downloads\you-get-0.4.1128.zip') zipimport.ZipImportError: not a Zip file: 'C:\\Users\\i\\Downloads\\you-get-0.4.1128.zip' >>> [1]: https://github.com/soimort/you-get/releases [2]: https://github.com/soimort/you-get/archive/v0.4.1128.zip From ntrewartha2 at gmail.com Mon Aug 13 12:37:19 2018 From: ntrewartha2 at gmail.com (Keep Secret) Date: Mon, 13 Aug 2018 09:37:19 -0700 (PDT) Subject: Python 3.6 Logging time is not listed Message-ID: #!/usr/bin/env python3 import logging logging.basicConfig(filename='example.log',level=logging.DEBUG) logging.basicConfig(format='%(asctime)s;%(levelname)s:%(message)s', level=logging.DEBUG) logging.debug('Message1) logging.info('Message2') logging.warning('Message3') DEBUG:root:Message1 INFO:root:Message2 WARNING:root:Message3 BUT if I remove logging.basicConfig(filename='example.log',level=logging.DEBUG) I get 2018-08-13 18:35:48,982;DEBUG:Message1 2018-08-13 18:35:48,982;INFO:Message2 2018-08-13 18:35:48,983;WARNING:Message3 Can someone please tell me what I am doing wrong? Thanks From abrault at mapgears.com Mon Aug 13 12:54:45 2018 From: abrault at mapgears.com (Alexandre Brault) Date: Mon, 13 Aug 2018 12:54:45 -0400 Subject: Python 3.6 Logging time is not listed In-Reply-To: References: Message-ID: On 2018-08-13 12:37 PM, Keep Secret wrote: > #!/usr/bin/env python3 > import logging > logging.basicConfig(filename='example.log',level=logging.DEBUG) > logging.basicConfig(format='%(asctime)s;%(levelname)s:%(message)s', level=logging.DEBUG) > logging.debug('Message1) > logging.info('Message2') > logging.warning('Message3') > > DEBUG:root:Message1 > INFO:root:Message2 > WARNING:root:Message3 > BUT if I remove logging.basicConfig(filename='example.log',level=logging.DEBUG) > I get > 2018-08-13 18:35:48,982;DEBUG:Message1 > 2018-08-13 18:35:48,982;INFO:Message2 > 2018-08-13 18:35:48,983;WARNING:Message3 > > Can someone please tell me what I am doing wrong? > Thanks The logging module doesn't allow you to update its config. You're supposed to configure all the options you want in a single call to basicConfig (or one of the other config methods for a different way to configure logging) >>> import logging >>> logging.basicConfig(filename='example.log',level=logging.DEBUG,format='%(asctime)s;%(levelname)s:%(message)s') >>> logging.info('Message2') >>> logging.warning('Message3') produces this 2018-08-13 12:52:03,330;INFO:Message2 2018-08-13 12:52:04,231;WARNING:Message3 Alex From python at mrabarnett.plus.com Mon Aug 13 13:23:53 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 13 Aug 2018 18:23:53 +0100 Subject: Python 3.6 Logging time is not listed In-Reply-To: References: Message-ID: <4c7460f1-a5c4-0a7d-4db5-84ea5480c489@mrabarnett.plus.com> On 2018-08-13 17:37, Keep Secret wrote: > #!/usr/bin/env python3 > import logging Here you're configuring the logger, setting the name of the logfile and the logging level, but not specifying the format, so it uses the default format: > logging.basicConfig(filename='example.log',level=logging.DEBUG) Here you're configuring the logger again, this time specifying the format and the logging level, but not a path of a logging file, so it'll write to the console: > logging.basicConfig(format='%(asctime)s;%(levelname)s:%(message)s', level=logging.DEBUG) The second configuration is overriding the first. > logging.debug('Message1) > logging.info('Message2') > logging.warning('Message3') > > DEBUG:root:Message1 > INFO:root:Message2 > WARNING:root:Message3 > BUT if I remove logging.basicConfig(filename='example.log',level=logging.DEBUG) > I get > 2018-08-13 18:35:48,982;DEBUG:Message1 > 2018-08-13 18:35:48,982;INFO:Message2 > 2018-08-13 18:35:48,983;WARNING:Message3 > > Can someone please tell me what I am doing wrong? > Thanks > From leo at superlel.me Mon Aug 13 13:42:33 2018 From: leo at superlel.me (=?UTF-8?Q?L=c3=a9o_El_Amri?=) Date: Mon, 13 Aug 2018 19:42:33 +0200 Subject: Python 3.6 Logging time is not listed In-Reply-To: <4c7460f1-a5c4-0a7d-4db5-84ea5480c489@mrabarnett.plus.com> References: <4c7460f1-a5c4-0a7d-4db5-84ea5480c489@mrabarnett.plus.com> Message-ID: <7f9b776a-35f5-f679-5b69-d804c194fa18@superlel.me> On 13/08/2018 19:23, MRAB wrote: > Here you're configuring the logger, setting the name of the logfile and > the logging level, but not specifying the format, so it uses the default > format: > >> logging.basicConfig(filename='example.log',level=logging.DEBUG) > > Here you're configuring the logger again, this time specifying the > format and the logging level, but not a path of a logging file, so it'll > write to the console: > >> logging.basicConfig(format='%(asctime)s;%(levelname)s:%(message)s', >> level=logging.DEBUG) > > The second configuration is overriding the first. No, the second is not overriding the first one. The second call simply does nothing at all. See https://docs.python.org/3/library/logging.html#logging.basicConfig : "This function does nothing if the root logger already has handlers configured for it." From arj.python at gmail.com Mon Aug 13 14:43:00 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 13 Aug 2018 22:43:00 +0400 Subject: wiki.python.org Gateway time-out Message-ID: creating an account on wiki.python.org, always same error : gateway time-out. any solution? yours, Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius From skybuck2000 at hotmail.com Mon Aug 13 15:54:53 2018 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Mon, 13 Aug 2018 12:54:53 -0700 (PDT) Subject: >< swap operator Message-ID: Hello, I just had a funny idea how to implement a swap operator for types: A >< B would mean swap A and B. Bye, Skybuck. From leo at superlel.me Mon Aug 13 16:01:19 2018 From: leo at superlel.me (=?UTF-8?Q?L=c3=a9o_El_Amri?=) Date: Mon, 13 Aug 2018 22:01:19 +0200 Subject: >< swap operator In-Reply-To: References: Message-ID: <721c1745-c89d-9d5b-8ea3-66d040a2c262@superlel.me> On 13/08/2018 21:54, skybuck2000 at hotmail.com wrote: > I just had a funny idea how to implement a swap operator for types: > > A >< B > > would mean swap A and B. I think that: a, b = b, a is pretty enough From dieter at handshake.de Tue Aug 14 01:10:52 2018 From: dieter at handshake.de (dieter) Date: Tue, 14 Aug 2018 07:10:52 +0200 Subject: right way to use zipimport, zipimport.ZipImportError: not a Zip file References: <7aa781a2-a404-4d3c-862a-1de8afe211ba@googlegroups.com> Message-ID: <87ftzhjzxf.fsf@handshake.de> iMath writes: > ... > It seems possible in Python to use `zipimport` to directly use the zip archive of the module without extraction, I wonder what is the right way to use `zipimport`. Your problem may come from the following restriction (cited from the Python 2 documentation of `zipimport`): ZIP archives with an archive comment are currently not supported. This implies that "zipimport" cannot handle all (valid) zip archives. Another citation from the documentation mentioned above: This module adds the ability to import Python modules (*.py, *.py[co]) and packages from ZIP-format archives. It is usually not needed to use the zipimport module explicitly; it is automatically used by the built-in import mechanism for sys.path items that are paths to ZIP archives. This means, you usually do not use "zipimport" yourself. Instead, you ensure that `sys.path` contains a path leading to a zip archive or a directory inside a zip archive. I have used this feature in the past - and it works for supported zip archives. From redstone-cold at 163.com Tue Aug 14 03:34:17 2018 From: redstone-cold at 163.com (iMath) Date: Tue, 14 Aug 2018 00:34:17 -0700 (PDT) Subject: right way to use zipimport, zipimport.ZipImportError: not a Zip file In-Reply-To: References: <7aa781a2-a404-4d3c-862a-1de8afe211ba@googlegroups.com> <87ftzhjzxf.fsf@handshake.de> Message-ID: <94fb53dc-016e-4ed0-8206-05ea1e61d03d@googlegroups.com> I think someone gives the true reason caused the exception here https://stackoverflow.com/a/51821910/1485853 Thanks to his explanation , I extracted the zip archive and then add the extracted to a zip archive using Bandizip, this time `zipimport.zipimporter(r'C:\Users\i\Downloads\you-get-0.4.1128.zip') ` doesn't give the exception , but still cannot import the module, even adding the `.zip` file to `sys.path`, >>> import sys >>> sys.path.insert(0, r'C:\Users\i\Downloads\you-get-0.4.1128.zip\you-get-0.4.1128\src') >>> from you_get import common Traceback (most recent call last): File "", line 1, in from you_get import common ModuleNotFoundError: No module named 'you_get' >>> >>> import zipimport >>> z=zipimport.zipimporter(r'C:\Users\i\Downloads\you-get-0.4.1128.zip\you-get-0.4.1128\src') >>> z >>> z.load_module('you_get.common') Traceback (most recent call last): File "", line 1, in z.load_module('you_get.common') zipimport.ZipImportError: can't find module 'you_get.common' >>> z.load_module('you_get') Traceback (most recent call last): File "", line 1, in z.load_module('you_get') zipimport.ZipImportError: can't find module 'you_get' What I actually want to do is to use the module in a pyinstaller frozen application , I also need to upgrade the module to latest version whenever needed , I cannot find a solution yet. From redstone-cold at 163.com Tue Aug 14 03:34:25 2018 From: redstone-cold at 163.com (iMath) Date: Tue, 14 Aug 2018 00:34:25 -0700 (PDT) Subject: right way to use zipimport, zipimport.ZipImportError: not a Zip file In-Reply-To: References: <7aa781a2-a404-4d3c-862a-1de8afe211ba@googlegroups.com> <87ftzhjzxf.fsf@handshake.de> Message-ID: I think someone gives the true reason caused the exception here https://stackoverflow.com/a/51821910/1485853 Thanks to his explanation , I extracted the zip archive and then add the extracted to a zip archive using Bandizip, this time `zipimport.zipimporter(r'C:\Users\i\Downloads\you-get-0.4.1128.zip') ` doesn't give the exception , but still cannot import the module, even adding the `.zip` file to `sys.path`, >>> import sys >>> sys.path.insert(0, r'C:\Users\i\Downloads\you-get-0.4.1128.zip\you-get-0.4.1128\src') >>> from you_get import common Traceback (most recent call last): File "", line 1, in from you_get import common ModuleNotFoundError: No module named 'you_get' >>> >>> import zipimport >>> z=zipimport.zipimporter(r'C:\Users\i\Downloads\you-get-0.4.1128.zip\you-get-0.4.1128\src') >>> z >>> z.load_module('you_get.common') Traceback (most recent call last): File "", line 1, in z.load_module('you_get.common') zipimport.ZipImportError: can't find module 'you_get.common' >>> z.load_module('you_get') Traceback (most recent call last): File "", line 1, in z.load_module('you_get') zipimport.ZipImportError: can't find module 'you_get' What I actually want to do is to use the module in a pyinstaller frozen application , I also need to upgrade the module to latest version whenever needed , I cannot find a solution yet. From frank at chagford.com Tue Aug 14 03:38:53 2018 From: frank at chagford.com (Frank Millman) Date: Tue, 14 Aug 2018 09:38:53 +0200 Subject: Pylint false positives Message-ID: Hi all Pylint is flagging a lot of lines as errors that I would consider to be acceptable. I have an abstract class ClassA with a number of concrete sub-classes. ClassA has a method which invokes 'self.method_b()' which is defined separately on each sub-class. Pylint complains that "Instance of 'ClassA' has no 'method_b' member". First question - as a matter of style, is Pylint correct? If so, I could define 'method_b' in ClassA and raise NotImplementedError. Is this considered more pythonic? The downside is that I have quite a few of them, so it would add some clutter. Second question - if my present code is not unpythonic, is there an easy way to suppress the error messages, without disabling 'no-member' altogether? Thanks Frank Millman From darcy at VybeNetworks.com Tue Aug 14 04:20:51 2018 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Tue, 14 Aug 2018 04:20:51 -0400 Subject: Pylint false positives In-Reply-To: References: Message-ID: <865ed61a-cf1d-959f-f77e-dc586fe6e0cf@VybeNetworks.com> On 2018-08-14 03:38 AM, Frank Millman wrote: > Hi all > > Pylint is flagging a lot of lines as errors that I would consider to be > acceptable. > > I have an abstract class ClassA with a number of concrete sub-classes. > ClassA has a method which invokes 'self.method_b()' which is defined > separately on each sub-class. Pylint complains that "Instance of > 'ClassA' has no? 'method_b' member". > > First question - as a matter of style, is Pylint correct? If so, I could > define 'method_b' in ClassA and raise NotImplementedError. Is this > considered more pythonic? The downside is that I have quite a few of > them, so it would add some clutter. I would add the method. It's one line: def method_b(self): raise NotImplementedError When you say that you have quite a lot of them, what is "them"? Many master classes or many methods in the class? If the latter, one line each isn't so bad. If the former I wonder if a master, master class is called for. I am also getting a funny smell from your description. Are you sure that you need to redefine the methods? Perhaps you just need to define some class variables and use one method. You can also define your own method and call the classA method inside it for common functionality. Just my 2? based on no information about your actual application. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From frank at chagford.com Tue Aug 14 04:58:17 2018 From: frank at chagford.com (Frank Millman) Date: Tue, 14 Aug 2018 10:58:17 +0200 Subject: Pylint false positives In-Reply-To: <865ed61a-cf1d-959f-f77e-dc586fe6e0cf@VybeNetworks.com> References: <865ed61a-cf1d-959f-f77e-dc586fe6e0cf@VybeNetworks.com> Message-ID: "D'Arcy Cain" wrote in message news:865ed61a-cf1d-959f-f77e-dc586fe6e0cf at VybeNetworks.com... > > On 2018-08-14 03:38 AM, Frank Millman wrote: > > Hi all > > > > Pylint is flagging a lot of lines as errors that I would consider to be > > acceptable. > > > > I have an abstract class ClassA with a number of concrete sub-classes. > > ClassA has a method which invokes 'self.method_b()' which is defined > > separately on each sub-class. Pylint complains that "Instance of > > 'ClassA' has no 'method_b' member". > > > > First question - as a matter of style, is Pylint correct? If so, I could > > define 'method_b' in ClassA and raise NotImplementedError. Is this > > considered more pythonic? The downside is that I have quite a few of > > them, so it would add some clutter. > > I would add the method. It's one line: > > def method_b(self): raise NotImplementedError > > When you say that you have quite a lot of them, what is "them"? Many > master classes or many methods in the class? If the latter, one line > each isn't so bad. If the former I wonder if a master, master class is > called for. > I do mean a lot of methods, not classes. I don't have any problem adding the lines. It is just that, before I starting using pylint, it had not occurred to me that there was any problem with my approach. If an experienced python programmer was reviewing my code, would they flag it as 'bad style'? > I am also getting a funny smell from your description. Are you sure > that you need to redefine the methods? Perhaps you just need to define > some class variables and use one method. You can also define your own > method and call the classA method inside it for common functionality. > As an example, I have a master class defining a unit of data (i.e. the value of a column) retrieved from a database. I have separate sub-classes for each data type - text, integer, date, etc. To ensure that a value is valid before storing it as an instance attribute, I call a method called 'check_value'. The details of check_value vary according to the data type, but that is transparent to the piece of code that calls check_value(). Frank From tjol at tjol.eu Tue Aug 14 05:05:48 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 14 Aug 2018 11:05:48 +0200 Subject: Pylint false positives In-Reply-To: References: Message-ID: <53faf0ef-4054-53fa-6179-a862495eadac@tjol.eu> On 2018-08-14 09:38, Frank Millman wrote: > Hi all > > Pylint is flagging a lot of lines as errors that I would consider to be > acceptable. > > I have an abstract class ClassA with a number of concrete sub-classes. > ClassA has a method which invokes 'self.method_b()' which is defined > separately on each sub-class. Pylint complains that "Instance of > 'ClassA' has no? 'method_b' member". > > First question - as a matter of style, is Pylint correct? If so, I could > define 'method_b' in ClassA and raise NotImplementedError. Is this > considered more pythonic? The downside is that I have quite a few of > them, so it would add some clutter. I wouldn't say it's unpythonic per se, but if your ClassA is logically an abstract base class that requires certain methods to be implemented in subclasses, then it's probably clearer to use Python's abc [1] facilities, and declare your abstract methods as actual abstractmethods. [1]: https://docs.python.org/3/library/abc.html i.e. from abc import ABC, abstractmethod class ClassA(ABC): def do_stuff(self): return self.method_b(42)**3 @abstractmethod def method_b(self, answer): """ This is a great place to put a docstring """ You *can* raise NotImplementedError in your abstractmethods, but I don't think it's really necessary. You need a statement in the method body of course, but since you're going to put a docstring there anyway (right?), that's already taken care of. -- Thomas > Second question - if my present code is not unpythonic, is there an easy > way to suppress the error messages, without disabling 'no-member' > altogether? From frank at chagford.com Tue Aug 14 05:54:39 2018 From: frank at chagford.com (Frank Millman) Date: Tue, 14 Aug 2018 11:54:39 +0200 Subject: Pylint false positives In-Reply-To: <53faf0ef-4054-53fa-6179-a862495eadac@tjol.eu> References: <53faf0ef-4054-53fa-6179-a862495eadac@tjol.eu> Message-ID: "Thomas Jollans" wrote in message news:53faf0ef-4054-53fa-6179-a862495eadac at tjol.eu... > > On 2018-08-14 09:38, Frank Millman wrote: > > Hi all > > > > Pylint is flagging a lot of lines as errors that I would consider to be > > acceptable. > > > > I have an abstract class ClassA with a number of concrete sub-classes. > > ClassA has a method which invokes 'self.method_b()' which is defined > > separately on each sub-class. Pylint complains that "Instance of > > 'ClassA' has no 'method_b' member". > > > > First question - as a matter of style, is Pylint correct? If so, I could > > define 'method_b' in ClassA and raise NotImplementedError. Is this > > considered more pythonic? The downside is that I have quite a few of > > them, so it would add some clutter. > > I wouldn't say it's unpythonic per se, but if your ClassA is logically > an abstract base class that requires certain methods to be implemented > in subclasses, then it's probably clearer to use Python's abc [1] > facilities, and declare your abstract methods as actual abstractmethods. > > > [1]: https://docs.python.org/3/library/abc.html > Thanks for the pointer - I will look into that alternative. [...] > > You *can* raise NotImplementedError in your abstractmethods, but I don't > think it's really necessary. You need a statement in the method body of > course, but since you're going to put a docstring there anyway (right?), > that's already taken care of. > At first I thought that it would be necessary, but then I saw in the docs that an ABC class "cannot be instantiated unless all of its abstract methods and properties are overridden", so there is no danger of forgetting to add it to the subclass. Frank From skybuck2000 at hotmail.com Tue Aug 14 09:18:41 2018 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Tue, 14 Aug 2018 06:18:41 -0700 (PDT) Subject: >< swap operator In-Reply-To: References: <721c1745-c89d-9d5b-8ea3-66d040a2c262@superlel.me> Message-ID: <66a7ecac-0098-4166-b273-979c6bdab99a@googlegroups.com> On Monday, August 13, 2018 at 10:01:37 PM UTC+2, L?o El Amri wrote: > On 13/08/2018 21:54, skybuck2000 at hotmail.com wrote: > > I just had a funny idea how to implement a swap operator for types: > > > > A >< B > > > > would mean swap A and B. > > I think that: > > a, b = b, a > > is pretty enough LOL. A >< B is shorter ! And PPPython is all about writing short and lazy code ! =D (>< reminds me of XOR swap ;)) Bye, Skybuck ;) From marko at pacujo.net Tue Aug 14 09:28:19 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 14 Aug 2018 16:28:19 +0300 Subject: >< swap operator References: <721c1745-c89d-9d5b-8ea3-66d040a2c262@superlel.me> <66a7ecac-0098-4166-b273-979c6bdab99a@googlegroups.com> Message-ID: <8736vhnklo.fsf@elektro.pacujo.net> skybuck2000 at hotmail.com: > On Monday, August 13, 2018 at 10:01:37 PM UTC+2, L?o El Amri wrote: >> On 13/08/2018 21:54, skybuck2000 at hotmail.com wrote: >> > I just had a funny idea how to implement a swap operator for types: >> > >> > A >< B >> > >> > would mean swap A and B. >> >> I think that: >> >> a, b = b, a >> >> is pretty enough > > LOL. > > A >< B is shorter ! Don't forget the (A >:< B) operator: def fib(n): curr = prev = 1 for _ in range(n): (curr >:< prev) += prev return curr Obviously, we see immediately the need for the Pythonic improvement: def fib(n): curr = prev = 1 for _ in range(n): curr >+=< prev return curr Marko From steve+comp.lang.python at pearwood.info Tue Aug 14 09:41:41 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 14 Aug 2018 13:41:41 +0000 (UTC) Subject: Pylint false positives References: <865ed61a-cf1d-959f-f77e-dc586fe6e0cf@VybeNetworks.com> Message-ID: On Tue, 14 Aug 2018 10:58:17 +0200, Frank Millman wrote: >> > I have an abstract class ClassA with a number of concrete >> > sub-classes. ClassA has a method which invokes 'self.method_b()' >> > which is defined separately on each sub-class. Pylint complains that >> > "Instance of 'ClassA' has no 'method_b' member". [...] > I do mean a lot of methods, not classes. I don't have any problem adding > the lines. It is just that, before I starting using pylint, it had not > occurred to me that there was any problem with my approach. If an > experienced python programmer was reviewing my code, would they flag it > as 'bad style'? *shrug* I wouldn't necessarily call it *bad*, but perhaps *not-quite good* style. I think its fine for a small projects and quick scripts, especially if they're written and maintained by a single person for their own use. Perhaps not so much for large projects intended for long-term use with continual development. If there really are a lot of such missing methods, I'd consider writing something like this: class A: def __init__(self, ...): ... # === process abstract methods en masse === for name in "method_a method_b method_c method_d".split(): @abstractmethod def inner(self): raise NotImplementedError inner.__name__ = name # This is okay, writing to locals works inside the class body. locals()[name] = inner del inner, name # Clean up the class namespace. def concrete_method_a(self): ... although to be honest I'm not sure if that would be enough to stop PyLint from complaining. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson From brian.j.oney at googlemail.com Tue Aug 14 11:07:37 2018 From: brian.j.oney at googlemail.com (Brian Oney) Date: Tue, 14 Aug 2018 17:07:37 +0200 Subject: >< swap operator In-Reply-To: <12r5nd9865hbdg7b578cqn089gn7paslnj@4ax.com> References: <721c1745-c89d-9d5b-8ea3-66d040a2c262@superlel.me> <66a7ecac-0098-4166-b273-979c6bdab99a@googlegroups.com> <12r5nd9865hbdg7b578cqn089gn7paslnj@4ax.com> Message-ID: <1534259257.10240.1.camel@gmail.com> On Tue, 2018-08-14 at 10:55 -0400, Dennis Lee Bieber wrote: > On Tue, 14 Aug 2018 06:18:41 -0700 (PDT), skybuck2000 at hotmail.com declaimed > the following: > > > On Monday, August 13, 2018 at 10:01:37 PM UTC+2, L?o El Amri wrote: > > > On 13/08/2018 21:54, skybuck2000 at hotmail.com wrote: > > > > I just had a funny idea how to implement a swap operator for types: > > > > > > > > A >< B > > > > > > > > would mean swap A and B. > > > > > > I think that: > > > > > > a, b = b, a > > > > > > is pretty enough > > > > LOL. > > > > A >< B is shorter ! > > > > But is specific to swapping just two entities... > > c, a, b = a, b, c > > moves three entities at once -- and can be expanded for more. > > > > > a = 1 > > > > b = 2 > > > > c = "confusion" > > > > > > > > c, a, b = a, b, c > > > > print a > > 2 > > > > print b > > confusion > > > > print c > > 1 Indeed, that is elegant. It doesn't take a special knowledge of syntax, for good guessers. It also doesn't look like two male variables peeing on each others feet. From jon+usenet at unequivocal.eu Tue Aug 14 11:18:13 2018 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 14 Aug 2018 15:18:13 -0000 (UTC) Subject: Pylint false positives References: <865ed61a-cf1d-959f-f77e-dc586fe6e0cf@VybeNetworks.com> Message-ID: On 2018-08-14, Steven D'Aprano wrote: > If there really are a lot of such missing methods, I'd consider writing > something like this: > > class A: > def __init__(self, ...): > ... > > # === process abstract methods en masse === > for name in "method_a method_b method_c method_d".split(): > @abstractmethod > def inner(self): > raise NotImplementedError > inner.__name__ = name > # This is okay, writing to locals works inside the class body. > locals()[name] = inner > > del inner, name # Clean up the class namespace. You have a peculiar idea of "good style"... > although to be honest I'm not sure if that would be enough to stop PyLint > from complaining. No - if you think about it, there's no way Pylint could possibly know that the above class has methods method_a, method_b, etc. It also doesn't like the `del inner, name` because theoretically neither of those names might be defined, if the loop executed zero times. From hjp-python at hjp.at Tue Aug 14 15:32:16 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Tue, 14 Aug 2018 21:32:16 +0200 Subject: Good reason not to obfuscate URLs (was: Fishing from PyPI ?) In-Reply-To: References: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> <86tvo6oguh.fsf@benfinney.id.au> Message-ID: <20180814193216.duoaossejvglhovx@hjp.at> On 2018-08-08 05:18:21 +0000, Gilmeh Serda wrote: > And if you read email in blasted HTML, chances are they also have an > image that they serve to you on their "beautiful" page you receive, an > image whose link which may or may not be equally personalized, and more > often than not has its origin on the spammer's server. in which case they > also know *exactly when* you opened the email. If at all. > > Oh, and thank the developer of Outlook, and similarly badly constructed > programs, for that preview pane. It's really helpful for all the spammers. I hate to defend Outlook (which I think is a really bad MUA), but it gets this one right: Properly configured[1] it does NOT load inline images from web-pages, so you can't be tracked simply by opening a mail. hp [1] Not sure whether this is the default or whether our admins configured it that way. -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ian.g.kelly at gmail.com Tue Aug 14 16:25:04 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 14 Aug 2018 14:25:04 -0600 Subject: Good reason not to obfuscate URLs (was: Fishing from PyPI ?) In-Reply-To: <20180814193216.duoaossejvglhovx@hjp.at> References: <9a4de78d-c2de-dadd-f76a-0bdecc031d34@telenet.be> <86tvo6oguh.fsf@benfinney.id.au> <20180814193216.duoaossejvglhovx@hjp.at> Message-ID: On Tue, Aug 14, 2018 at 1:41 PM Peter J. Holzer wrote: > > On 2018-08-08 05:18:21 +0000, Gilmeh Serda wrote: > > And if you read email in blasted HTML, chances are they also have an > > image that they serve to you on their "beautiful" page you receive, an > > image whose link which may or may not be equally personalized, and more > > often than not has its origin on the spammer's server. in which case they > > also know *exactly when* you opened the email. If at all. > > > > Oh, and thank the developer of Outlook, and similarly badly constructed > > programs, for that preview pane. It's really helpful for all the spammers. > > I hate to defend Outlook (which I think is a really bad MUA), but it > gets this one right: Properly configured[1] it does NOT load inline images > from web-pages, so you can't be tracked simply by opening a mail. > > hp > > [1] Not sure whether this is the default or whether our admins > configured it that way. Same for Gmail, I believe. It doesn't show the external images unless you explicitly click something to load them. I would expect any competent MUA of at least the past decade to do the same. From tjreedy at udel.edu Tue Aug 14 17:42:14 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 14 Aug 2018 17:42:14 -0400 Subject: Pylint false positives In-Reply-To: <53faf0ef-4054-53fa-6179-a862495eadac@tjol.eu> References: <53faf0ef-4054-53fa-6179-a862495eadac@tjol.eu> Message-ID: On 8/14/2018 5:05 AM, Thomas Jollans wrote: > On 2018-08-14 09:38, Frank Millman wrote: >> Hi all >> >> Pylint is flagging a lot of lines as errors that I would consider to be >> acceptable. >> >> I have an abstract class ClassA with a number of concrete sub-classes. >> ClassA has a method which invokes 'self.method_b()' which is defined >> separately on each sub-class. Pylint complains that "Instance of >> 'ClassA' has no? 'method_b' member". >> >> First question - as a matter of style, is Pylint correct? If so, I could >> define 'method_b' in ClassA and raise NotImplementedError. Is this >> considered more pythonic? The downside is that I have quite a few of >> them, so it would add some clutter. If I were reading you class, I would like to see all methods defined there. If you went the old NotImplemented(Error) route, you could avoid cluttter with method_c = method_d = method_e = method_b but I would also want docstrings. At that point, I would consider abstractmethod. But I have not used that, not seen an stdlib class that does. > I wouldn't say it's unpythonic per se, but if your ClassA is logically > an abstract base class that requires certain methods to be implemented > in subclasses, then it's probably clearer to use Python's abc [1] > facilities, and declare your abstract methods as actual abstractmethods. > > [1]: https://docs.python.org/3/library/abc.html > > i.e. > > from abc import ABC, abstractmethod > class ClassA(ABC): > def do_stuff(self): > return self.method_b(42)**3 > > @abstractmethod > def method_b(self, answer): > """ > This is a great place to put a docstring > """ > > You *can* raise NotImplementedError in your abstractmethods, but I don't > think it's really necessary. You need a statement in the method body of > course, but since you're going to put a docstring there anyway (right?), > that's already taken care of. > > -- Thomas > > >> Second question - if my present code is not unpythonic, is there an easy >> way to suppress the error messages, without disabling 'no-member' >> altogether? -- Terry Jan Reedy From python at bdurham.com Tue Aug 14 17:45:26 2018 From: python at bdurham.com (Malcolm Greene) Date: Tue, 14 Aug 2018 15:45:26 -0600 Subject: How to pass Python command line options (vs arguments) when running script directly vs via Python interpreter? Message-ID: <1534283126.3954352.1474277624.5C4E855F@webmail.messagingengine.com> When you run a script via "python3 script.py" you can include command line options like -b, -B, -O, -OO, etc between the "python3" interpreter reference and the script.py file, eg. "python3 -b -B -O -OO script.py". When you create a script that is executable directly, eg. script.py with execution bit set on Linux or on Windows where the .py file extension is associated with a specific Python executable, there doesn't appear to be a way to pass command line options to the script. In this later case, how can I pass my script command line options without having these options confused with command line arguments? Thank you, Malcolm From michael.stemper at gmail.com Tue Aug 14 18:05:30 2018 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Tue, 14 Aug 2018 17:05:30 -0500 Subject: How to pass Python command line options (vs arguments) when running script directly vs via Python interpreter? In-Reply-To: References: <1534283126.3954352.1474277624.5C4E855F@webmail.messagingengine.com> Message-ID: On 2018-08-14 16:45, Malcolm Greene wrote: > When you run a script via "python3 script.py" you can include command > line options like -b, -B, -O, -OO, etc between the "python3" interpreter > reference and the script.py file, eg. "python3 -b -B -O -OO script.py". > When you create a script that is executable directly, eg. script.py with > execution bit set on Linux or on Windows where the .py file extension is > associated with a specific Python executable, there doesn't appear to be > a way to pass command line options to the script. In this later case, > how can I pass my script command line options without having these > options confused with command line arguments? You might try: from getopt import getopt or the (apparently newer): from optparse import OptionParser These appear to both be deprecated in favor of argparse. I haven't made the switch myself because argparse appears (upon a cursory reading of the documentation) to muddle the difference between options and arguments. -- Michael F. Stemper Indians scattered on dawn's highway bleeding; Ghosts crowd the young child's fragile eggshell mind. From tjreedy at udel.edu Tue Aug 14 18:08:53 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 14 Aug 2018 18:08:53 -0400 Subject: How to pass Python command line options (vs arguments) when running script directly vs via Python interpreter? In-Reply-To: <1534283126.3954352.1474277624.5C4E855F@webmail.messagingengine.com> References: <1534283126.3954352.1474277624.5C4E855F@webmail.messagingengine.com> Message-ID: On 8/14/2018 5:45 PM, Malcolm Greene wrote: > When you run a script via "python3 script.py" you can include command > line options like -b, -B, -O, -OO, etc between the "python3" interpreter > reference and the script.py file, eg. "python3 -b -B -O -OO script.py". More generally, python script.py