From marko at pacujo.net  Sat Dec  1 05:10:50 2018
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 01 Dec 2018 12:10:50 +0200
Subject: multiple JSON documents in one file, change proposal
References: <87sgzijjjl.fsf@nightsong.com> <87tvjyyyt5.fsf@elektro.pacujo.net>
 <87h8fyjhod.fsf@nightsong.com>
Message-ID: <87lg59zhed.fsf@elektro.pacujo.net>

Paul Rubin <no.email at nospam.invalid>:

> Marko Rauhamaa <marko at pacujo.net> writes:
>> Having rejected different options (<URL:
>> https://en.wikipedia.org/wiki/JSON_streaming>), I settled with
>> terminating each JSON value with an ASCII NUL character, which is
>> illegal in JSON proper.
>
> Thanks, that Wikipedia article is helpful.  I'd prefer to not use stuff
> like NUL or RS because I like keeping the file human readable.  I might
> use netstring format (http://cr.yp.to/proto/netstrings.txt) but I'm even
> more convinced now that adding a streaming feature to the existing json
> module is the right way to do it.

We all have our preferences.

In my case, I need an explicit terminator marker to know when a JSON
value is complete. For example, if I should read from a socket:

   123

I can't yet parse it because there might be another digit coming. On the
other hand, the peer might not see any reason to send any further bytes
because "123" is all they wanted to send at the moment.

As for NUL, a control character that is illegal in all JSON contexts is
practical so the JSON chunks don't need to be escaped. An ASCII-esque
solution would be to pick ETX (= end of text). Unfortunately, a human
operator typing ETX (= ctrl-C) to terminate a JSON value will cause a
KeyboardInterrupt in many modern command-line interfaces.

It happens NUL (= ctrl-SPC = ctrl-@) is pretty easy to generate and
manipulate in editors and the command line.

The need for the format to be "typable" (and editable) is essential for
ad-hoc manual testing of components. That precludes all framing formats
that would necessitate a length prefix. HTTP would be horrible to have
to type even without the content-length problem, but BEEP (RFC 3080)
would suffer from the content-length (and CRLF!) issue as well.

Finally, couldn't any whitespace character work as a terminator? Yes, it
could, but it would force you to use a special JSON parser that is
prepared to handle the self-delineation. A NUL gives you many more
degrees of freedom in choosing your JSON tools.


Marko


From rosuav at gmail.com  Sat Dec  1 05:30:53 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 1 Dec 2018 21:30:53 +1100
Subject: multiple JSON documents in one file, change proposal
In-Reply-To: <87lg59zhed.fsf@elektro.pacujo.net>
References: <87sgzijjjl.fsf@nightsong.com> <87tvjyyyt5.fsf@elektro.pacujo.net>
 <87h8fyjhod.fsf@nightsong.com> <87lg59zhed.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmqW3WoF0yTL2Ei4-hPUr74ffiYN49Bcxu4-LAhNMgnRxA@mail.gmail.com>

On Sat, Dec 1, 2018 at 9:16 PM Marko Rauhamaa <marko at pacujo.net> wrote:
>
> Paul Rubin <no.email at nospam.invalid>:
>
> > Marko Rauhamaa <marko at pacujo.net> writes:
> >> Having rejected different options (<URL:
> >> https://en.wikipedia.org/wiki/JSON_streaming>), I settled with
> >> terminating each JSON value with an ASCII NUL character, which is
> >> illegal in JSON proper.
> >
> > Thanks, that Wikipedia article is helpful.  I'd prefer to not use stuff
> > like NUL or RS because I like keeping the file human readable.  I might
> > use netstring format (http://cr.yp.to/proto/netstrings.txt) but I'm even
> > more convinced now that adding a streaming feature to the existing json
> > module is the right way to do it.
>
> We all have our preferences.
>
> In my case, I need an explicit terminator marker to know when a JSON
> value is complete. For example, if I should read from a socket:
>
>    123
>
> I can't yet parse it because there might be another digit coming. On the
> other hand, the peer might not see any reason to send any further bytes
> because "123" is all they wanted to send at the moment.

This is actually the only special case. Every other JSON value has a
clear end. So the only thing you need to say is that, if the sender
wishes to transmit a bare number, it must append a space. Seriously,
how often do you ACTUALLY send a bare number? I've sometimes sent a
string on its own, but even that is incredibly rare. Having to send a
simple space after a bare number is unlikely to cause much trouble.

> As for NUL, a control character that is illegal in all JSON contexts is
> practical so the JSON chunks don't need to be escaped. An ASCII-esque
> solution would be to pick ETX (= end of text). Unfortunately, a human
> operator typing ETX (= ctrl-C) to terminate a JSON value will cause a
> KeyboardInterrupt in many modern command-line interfaces.
>
> It happens NUL (= ctrl-SPC = ctrl-@) is pretty easy to generate and
> manipulate in editors and the command line.

I have no idea which editors YOU use, but if you poll across platforms
and systems, I'm pretty sure you'll find that not everyone can type
it. Furthermore, many tools use the presence of an 0x00 byte as
evidence that a file is binary, not text. (For instance, git does
this.) That might be a good choice for your personal use-case, but not
the general case, whereas the much simpler options listed on the
Wikipedia page are far more general, and actually wouldn't be THAT
hard for you to use.

> The need for the format to be "typable" (and editable) is essential for
> ad-hoc manual testing of components. That precludes all framing formats
> that would necessitate a length prefix. HTTP would be horrible to have
> to type even without the content-length problem, but BEEP (RFC 3080)
> would suffer from the content-length (and CRLF!) issue as well.

I dunno, I type HTTP manually often enough that it can't be all *that* horrible.

> Finally, couldn't any whitespace character work as a terminator? Yes, it
> could, but it would force you to use a special JSON parser that is
> prepared to handle the self-delineation. A NUL gives you many more
> degrees of freedom in choosing your JSON tools.

Either non-delimited or newline-delimited JSON is supported in a lot
of tools. I'm quite at a loss here as to how an unprintable character
gives you more freedom.

I get it: you have a bizarre set of tools and the normal solutions
don't work for you. But you can't complain about the tools not
supporting your use-cases. Just code up your own styles of doing
things that are unique to you.

ChrisA


From marko at pacujo.net  Sat Dec  1 06:13:08 2018
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 01 Dec 2018 13:13:08 +0200
Subject: multiple JSON documents in one file, change proposal
References: <87sgzijjjl.fsf@nightsong.com> <87tvjyyyt5.fsf@elektro.pacujo.net>
 <87h8fyjhod.fsf@nightsong.com> <87lg59zhed.fsf@elektro.pacujo.net>
 <CAPTjJmqW3WoF0yTL2Ei4-hPUr74ffiYN49Bcxu4-LAhNMgnRxA@mail.gmail.com>
 <mailman.43.1543660267.25456.python-list@python.org>
Message-ID: <87ftvhzeij.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:
> On Sat, Dec 1, 2018 at 9:16 PM Marko Rauhamaa <marko at pacujo.net> wrote:
>> The need for the format to be "typable" (and editable) is essential
>> for ad-hoc manual testing of components. That precludes all framing
>> formats that would necessitate a length prefix. HTTP would be
>> horrible to have to type even without the content-length problem, but
>> BEEP (RFC 3080) would suffer from the content-length (and CRLF!)
>> issue as well.
>
> I dunno, I type HTTP manually often enough that it can't be all *that*
> horrible.

Say I want to send this piece of JSON:

   {
       "msgtype": "echo-req",
       "opid": 3487547843
   }

and the framing format is HTTP. I will need to type something like this:

   POST / HTTP/1.1^M
   Host: localhost^M
   Content-type: application/json^M
   Content-length: 54^M
   ^M
   {
       "msgtype": "echo-req",
       "opid": 3487547843
   }

That's almost impossible to type without a syntax error.

>> Finally, couldn't any whitespace character work as a terminator? Yes,
>> it could, but it would force you to use a special JSON parser that is
>> prepared to handle the self-delineation. A NUL gives you many more
>> degrees of freedom in choosing your JSON tools.
>
> Either non-delimited or newline-delimited JSON is supported in a lot
> of tools. I'm quite at a loss here as to how an unprintable character
> gives you more freedom.

As stated by Paul in another context, newline-delimited is a no-go
because it forces you to restrict JSON to a subset that doesn't contain
newlines (see the JSON example above).

Of course, you could say that the terminating newline is only
interpreted as a terminator after a complete JSON value, but that's not
the format "supported in a lot of tools".

If you use any legal JSON character as a terminator, you have to make it
contextual or add an escaping syntax.

> I get it: you have a bizarre set of tools and the normal solutions
> don't work for you. But you can't complain about the tools not
> supporting your use-cases. Just code up your own styles of doing
> things that are unique to you.

There are numerous tools that parse complete JSON documents fine.
Framing JSON values with NUL-termination is trivial to add in any
programming environment. For example:

   def json_docs(path):
       with open(path) as f:
           for doc in f.read().split("\0")[:-1].
               yield json.loads(doc)


Marko


From rosuav at gmail.com  Sat Dec  1 06:28:21 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 1 Dec 2018 22:28:21 +1100
Subject: multiple JSON documents in one file, change proposal
In-Reply-To: <87ftvhzeij.fsf@elektro.pacujo.net>
References: <87sgzijjjl.fsf@nightsong.com> <87tvjyyyt5.fsf@elektro.pacujo.net>
 <87h8fyjhod.fsf@nightsong.com> <87lg59zhed.fsf@elektro.pacujo.net>
 <CAPTjJmqW3WoF0yTL2Ei4-hPUr74ffiYN49Bcxu4-LAhNMgnRxA@mail.gmail.com>
 <mailman.43.1543660267.25456.python-list@python.org>
 <87ftvhzeij.fsf@elektro.pacujo.net>
Message-ID: <CAPTjJmpFWw=gZ+qe4ThimKF_tRWO-o2DOAJAALpk+LXCrUN-yQ@mail.gmail.com>

On Sat, Dec 1, 2018 at 10:16 PM Marko Rauhamaa <marko at pacujo.net> wrote:
>
> Chris Angelico <rosuav at gmail.com>:
> > On Sat, Dec 1, 2018 at 9:16 PM Marko Rauhamaa <marko at pacujo.net> wrote:
> >> The need for the format to be "typable" (and editable) is essential
> >> for ad-hoc manual testing of components. That precludes all framing
> >> formats that would necessitate a length prefix. HTTP would be
> >> horrible to have to type even without the content-length problem, but
> >> BEEP (RFC 3080) would suffer from the content-length (and CRLF!)
> >> issue as well.
> >
> > I dunno, I type HTTP manually often enough that it can't be all *that*
> > horrible.
>
> Say I want to send this piece of JSON:
>
>    {
>        "msgtype": "echo-req",
>        "opid": 3487547843
>    }
>
> and the framing format is HTTP. I will need to type something like this:
>
>    POST / HTTP/1.1^M
>    Host: localhost^M
>    Content-type: application/json^M
>    Content-length: 54^M
>    ^M
>    {
>        "msgtype": "echo-req",
>        "opid": 3487547843
>    }
>
> That's almost impossible to type without a syntax error.

1) Set your Enter key to send CR-LF, at least for this operation.
That's half your problem solved.
2) Send the request like this:

POST / HTTP/1.0
Content-type: application/json

{"msgtype": "echo-req", "opid": 3487547843}

Then shut down your end of the connection, probably with Ctrl-D. I'm
fairly sure I can type that without bugs, and any compliant HTTP
server should be fine with it.

> >> Finally, couldn't any whitespace character work as a terminator? Yes,
> >> it could, but it would force you to use a special JSON parser that is
> >> prepared to handle the self-delineation. A NUL gives you many more
> >> degrees of freedom in choosing your JSON tools.
> >
> > Either non-delimited or newline-delimited JSON is supported in a lot
> > of tools. I'm quite at a loss here as to how an unprintable character
> > gives you more freedom.
>
> As stated by Paul in another context, newline-delimited is a no-go
> because it forces you to restrict JSON to a subset that doesn't contain
> newlines (see the JSON example above).
>
> Of course, you could say that the terminating newline is only
> interpreted as a terminator after a complete JSON value, but that's not
> the format "supported in a lot of tools".

The subset in question is simply "JSON without any newlines between
tokens", which has the exact meaning as it would have *with* those
newlines. So what you lose is the human-readability of being able to
break an object over multiple lines. Is that a problem? Use
non-delimited instead.

> If you use any legal JSON character as a terminator, you have to make it
> contextual or add an escaping syntax.

Or just use non-delimited, strip all whitespace between objects, and
then special-case the one otherwise-ambiguous situation of two Numbers
back to back. Anything that sends newline-delimited JSON will work
with that.

> > I get it: you have a bizarre set of tools and the normal solutions
> > don't work for you. But you can't complain about the tools not
> > supporting your use-cases. Just code up your own styles of doing
> > things that are unique to you.
>
> There are numerous tools that parse complete JSON documents fine.
> Framing JSON values with NUL-termination is trivial to add in any
> programming environment. For example:
>
>    def json_docs(path):
>        with open(path) as f:
>            for doc in f.read().split("\0")[:-1].
>                yield json.loads(doc)

Yes, but many text-processing tools don't let you manually insert
NULs. Of *course* you can put anything you like in there when you
control both ends and everything in between; that's kinda the point of
coding. But I'm going to use newlines, and parse as non-delimited,
since that can be done just as easily (see my example code earlier -
it could be converted into the same style of generator as you have
here and would be about as many lines), since that will behave as text
in most applications.

ChrisA


From marko at pacujo.net  Sat Dec  1 11:31:21 2018
From: marko at pacujo.net (Marko Rauhamaa)
Date: Sat, 01 Dec 2018 18:31:21 +0200
Subject: multiple JSON documents in one file, change proposal
References: <87sgzijjjl.fsf@nightsong.com> <87tvjyyyt5.fsf@elektro.pacujo.net>
 <87h8fyjhod.fsf@nightsong.com> <87lg59zhed.fsf@elektro.pacujo.net>
 <CAPTjJmqW3WoF0yTL2Ei4-hPUr74ffiYN49Bcxu4-LAhNMgnRxA@mail.gmail.com>
 <mailman.43.1543660267.25456.python-list@python.org>
 <87ftvhzeij.fsf@elektro.pacujo.net>
 <CAPTjJmpFWw=gZ+qe4ThimKF_tRWO-o2DOAJAALpk+LXCrUN-yQ@mail.gmail.com>
 <mailman.44.1543663715.25456.python-list@python.org>
Message-ID: <87a7lpyzs6.fsf@elektro.pacujo.net>

Chris Angelico <rosuav at gmail.com>:

> On Sat, Dec 1, 2018 at 10:16 PM Marko Rauhamaa <marko at pacujo.net> wrote:
>> and the framing format is HTTP. I will need to type something like this:
>>
>>    POST / HTTP/1.1^M
>>    Host: localhost^M
>>    Content-type: application/json^M
>>    Content-length: 54^M
>>    ^M
>>    {
>>        "msgtype": "echo-req",
>>        "opid": 3487547843
>>    }
>>
>> That's almost impossible to type without a syntax error.
>
> 1) Set your Enter key to send CR-LF, at least for this operation.
> That's half your problem solved.

That can be much harder than typing ctrl-SPC. It *is* supported by
netcat, for example, but then you have to carefully recompute the
content-length field.

> 2) Send the request like this:
>
> POST / HTTP/1.0
> Content-type: application/json
>
> {"msgtype": "echo-req", "opid": 3487547843}
>
> Then shut down your end of the connection, probably with Ctrl-D. I'm
> fairly sure I can type that without bugs, and any compliant HTTP
> server should be fine with it.

If I terminated the input, I wouldn't need any framing. The point is to
carry a number of JSON messages/documents over a single bytestream or in
a single file. That means the HTTP content-length would be mandatory.


Marko


From ben.usenet at bsb.me.uk  Sat Dec  1 13:50:00 2018
From: ben.usenet at bsb.me.uk (Ben Bacarisse)
Date: Sat, 01 Dec 2018 18:50:00 +0000
Subject: multiple JSON documents in one file, change proposal
References: <87sgzijjjl.fsf@nightsong.com> <87tvjyyyt5.fsf@elektro.pacujo.net>
 <87h8fyjhod.fsf@nightsong.com> <87lg59zhed.fsf@elektro.pacujo.net>
 <CAPTjJmqW3WoF0yTL2Ei4-hPUr74ffiYN49Bcxu4-LAhNMgnRxA@mail.gmail.com>
 <mailman.43.1543660267.25456.python-list@python.org>
 <87ftvhzeij.fsf@elektro.pacujo.net>
 <CAPTjJmpFWw=gZ+qe4ThimKF_tRWO-o2DOAJAALpk+LXCrUN-yQ@mail.gmail.com>
 <mailman.44.1543663715.25456.python-list@python.org>
 <87a7lpyzs6.fsf@elektro.pacujo.net>
Message-ID: <87muppf5ev.fsf@bsb.me.uk>

Marko Rauhamaa <marko at pacujo.net> writes:

> Chris Angelico <rosuav at gmail.com>:
>
>> On Sat, Dec 1, 2018 at 10:16 PM Marko Rauhamaa <marko at pacujo.net> wrote:
>>> and the framing format is HTTP. I will need to type something like this:
>>>
>>>    POST / HTTP/1.1^M
>>>    Host: localhost^M
>>>    Content-type: application/json^M
>>>    Content-length: 54^M
>>>    ^M
>>>    {
>>>        "msgtype": "echo-req",
>>>        "opid": 3487547843
>>>    }
>>>
>>> That's almost impossible to type without a syntax error.
>>
>> 1) Set your Enter key to send CR-LF, at least for this operation.
>> That's half your problem solved.
>
> That can be much harder than typing ctrl-SPC. It *is* supported by
> netcat, for example, but then you have to carefully recompute the
> content-length field.
>
>> 2) Send the request like this:
>>
>> POST / HTTP/1.0
>> Content-type: application/json
>>
>> {"msgtype": "echo-req", "opid": 3487547843}
>>
>> Then shut down your end of the connection, probably with Ctrl-D. I'm
>> fairly sure I can type that without bugs, and any compliant HTTP
>> server should be fine with it.
>
> If I terminated the input, I wouldn't need any framing. The point is to
> carry a number of JSON messages/documents over a single bytestream or in
> a single file. That means the HTTP content-length would be mandatory.

I haven't been following the details of the thread, but I wonder if a
multi-part form-data POST would be useful.  Way more complex than a
simple separator, of course, but more "standard" (for some rather vague
meaning of the term).

-- 
Ben.


From grant.b.edwards at gmail.com  Sat Dec  1 15:17:49 2018
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Sat, 1 Dec 2018 20:17:49 +0000 (UTC)
Subject: multiple JSON documents in one file, change proposal
References: <87sgzijjjl.fsf@nightsong.com> <87tvjyyyt5.fsf@elektro.pacujo.net>
Message-ID: <ptuq9d$m84$1@blaine.gmane.org>

On 2018-11-30, Marko Rauhamaa <marko at pacujo.net> wrote:
> Paul Rubin <no.email at nospam.invalid>:
>> Maybe someone can convince me I'm misusing JSON but I often want to
>> write out a file containing multiple records, and it's convenient to
>> use JSON to represent the record data.
>>
>> The obvious way to read a JSON doc from a file is with "json.load(f)"
>> where f is a file handle. Unfortunately, this throws an exception
>
> I have this "multi-JSON" need quite often. In particular, I exchange
> JSON-encoded messages over byte stream connections. There are many ways
> of doing it. Having rejected different options (<URL:
> https://en.wikipedia.org/wiki/JSON_streaming>), I settled with
> terminating each JSON value with an ASCII NUL character, which is
> illegal in JSON proper.

This is what "archive" file formats are for.  Just use tar, zip, ar,
cpio, or some other file format designed to store multiple "files" of
arbitrary data -- there are plenty of "standard" formats to choose
from and plenty of libraries to deal with them.

-- 
Grant



From morphex at gmail.com  Sat Dec  1 19:55:30 2018
From: morphex at gmail.com (Morten W. Petersen)
Date: Sun, 2 Dec 2018 01:55:30 +0100
Subject: ValueError vs IndexError, unpacking arguments with string.split
In-Reply-To: <CAPTjJmp9tbt-6nnR=FErQupPEWv12F3B=yhz+MuuV56Tkhnh2g@mail.gmail.com>
References: <CAEtLV=qGJSZ8z+Z7Wqtnp0ckdCvhK_LNdYbcgOVgso_cvM5VXQ@mail.gmail.com>
 <e22b2254-5f88-3ab3-9663-a5abf1826cf4@potatochowder.com>
 <CAEtLV=rfpTYYKXH0A0+DyEOwSQN8Xmo3kU4oTOGCnUwx3xfMBg@mail.gmail.com>
 <792be86a-d5bc-c66d-393e-68ef9fbfe4e0@potatochowder.com>
 <CAEtLV=r62+MduJdzDp-HHbqAgOdB91r2PMF2XHDj4rPrKOjtxw@mail.gmail.com>
 <d8bf19c8-8f2f-0a76-ddcf-bb292ad603d5@potatochowder.com>
 <CAEtLV=qATV9xX1qW2u0g2deTE04nBT35LBODpp4ENeniM08_og@mail.gmail.com>
 <CAPTjJmrAsO8fsNU86pHWXyCT4eXBhRiGDwho3NgZTNSta1y1Dw@mail.gmail.com>
 <CAEtLV=pE2-qVqh6rw3hAquaBDv=BL1xuBSUp=UgjjjwEwB3+FQ@mail.gmail.com>
 <CAPTjJmp9tbt-6nnR=FErQupPEWv12F3B=yhz+MuuV56Tkhnh2g@mail.gmail.com>
Message-ID: <CAEtLV=peq6aSTwnPGk6HJcstusoi+=0p5zJ56Y+SOW_tHkCzWA@mail.gmail.com>

On Sat, Dec 1, 2018 at 1:11 AM Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, Dec 1, 2018 at 10:55 AM Morten W. Petersen <morphex at gmail.com>
> wrote:
> > But this raises the question of how to write Python code,
> > short and sweet, that could handle infinite iterators in
> > such an unpack with multiple variables to assign to.
> >
> > Which I guess is mostly theoretical, as there are other
> > ways of designing code to avoid needing to unpack
> > an infinite iterator using the * prefix.
>
> It could only be done with the cooperation of the iterable in
> question. For instance, a range object could implement a "remove
> first" operation that does this, and several itertools types wouldn't
> need to change at all. But it can't be done generically other than the
> way it now is (pump the iterator the rest of the way).
>


I wasn't able to follow this, could you elaborate?

Regards,

Morten


-- 
Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue
Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen
Also playing music and podcasting here:
http://www.mixcloud.com/morten-w-petersen/
On Google+ here https://plus.google.com/107781930037068750156
On Instagram at https://instagram.com/morphexx/


From rosuav at gmail.com  Sat Dec  1 20:20:32 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 2 Dec 2018 12:20:32 +1100
Subject: ValueError vs IndexError, unpacking arguments with string.split
In-Reply-To: <CAEtLV=peq6aSTwnPGk6HJcstusoi+=0p5zJ56Y+SOW_tHkCzWA@mail.gmail.com>
References: <CAEtLV=qGJSZ8z+Z7Wqtnp0ckdCvhK_LNdYbcgOVgso_cvM5VXQ@mail.gmail.com>
 <e22b2254-5f88-3ab3-9663-a5abf1826cf4@potatochowder.com>
 <CAEtLV=rfpTYYKXH0A0+DyEOwSQN8Xmo3kU4oTOGCnUwx3xfMBg@mail.gmail.com>
 <792be86a-d5bc-c66d-393e-68ef9fbfe4e0@potatochowder.com>
 <CAEtLV=r62+MduJdzDp-HHbqAgOdB91r2PMF2XHDj4rPrKOjtxw@mail.gmail.com>
 <d8bf19c8-8f2f-0a76-ddcf-bb292ad603d5@potatochowder.com>
 <CAEtLV=qATV9xX1qW2u0g2deTE04nBT35LBODpp4ENeniM08_og@mail.gmail.com>
 <CAPTjJmrAsO8fsNU86pHWXyCT4eXBhRiGDwho3NgZTNSta1y1Dw@mail.gmail.com>
 <CAEtLV=pE2-qVqh6rw3hAquaBDv=BL1xuBSUp=UgjjjwEwB3+FQ@mail.gmail.com>
 <CAPTjJmp9tbt-6nnR=FErQupPEWv12F3B=yhz+MuuV56Tkhnh2g@mail.gmail.com>
 <CAEtLV=peq6aSTwnPGk6HJcstusoi+=0p5zJ56Y+SOW_tHkCzWA@mail.gmail.com>
Message-ID: <CAPTjJmrLrH4ODYLg2+TDqEfVmWnnCxpDwxVrFFFLOBv24qU3-g@mail.gmail.com>

On Sun, Dec 2, 2018 at 11:55 AM Morten W. Petersen <morphex at gmail.com> wrote:
>
> On Sat, Dec 1, 2018 at 1:11 AM Chris Angelico <rosuav at gmail.com> wrote:
>>
>> On Sat, Dec 1, 2018 at 10:55 AM Morten W. Petersen <morphex at gmail.com> wrote:
>> > But this raises the question of how to write Python code,
>> > short and sweet, that could handle infinite iterators in
>> > such an unpack with multiple variables to assign to.
>> >
>> > Which I guess is mostly theoretical, as there are other
>> > ways of designing code to avoid needing to unpack
>> > an infinite iterator using the * prefix.
>>
>> It could only be done with the cooperation of the iterable in
>> question. For instance, a range object could implement a "remove
>> first" operation that does this, and several itertools types wouldn't
>> need to change at all. But it can't be done generically other than the
>> way it now is (pump the iterator the rest of the way).
>
> I wasn't able to follow this, could you elaborate?
>

The way *x works in unpacking is that the entire iterable gets
unpacked, and everything gets put into a list that is then assigned to
x. This is generic, works on any iterable, but doesn't take advantage
of anything. Consider:

>>> x = range(3, 20, 5)
>>> first, *rest = x
>>> first
3
>>> rest
[8, 13, 18]

If a range object were asked to yield its first-and-rest in this way,
it could instead return range(8, 20, 5) - add the step onto the start,
job done. Same if you ask for some off the beginning and some off the
end And with a number of the itertools iterables/iterators, the "rest"
wouldn't actually need to change anything, since the iterator will get
consumed. This would need explicit support from the iterable, though,
as Python can't know how to do this generically; so there would need
to be a protocol for "unpack".

ChrisA


From akkana at shallowsky.com  Sat Dec  1 21:09:33 2018
From: akkana at shallowsky.com (Akkana Peck)
Date: Sat, 1 Dec 2018 19:09:33 -0700
Subject: multiple JSON documents in one file, change proposal
In-Reply-To: <20181130225012.GA92644@cskk.homeip.net>
References: <87tvjyyyt5.fsf@elektro.pacujo.net>
 <20181130225012.GA92644@cskk.homeip.net>
Message-ID: <20181202020933.GB1154@shallowsky.com>

Grant Edwards <grant.b.edwards at gmail.com> writes:
> This is what "archive" file formats are for.  Just use tar, zip, ar,
> cpio, or some other file format designed to store multiple "files" of
> arbitrary data -- there are plenty of "standard" formats to choose
> from and plenty of libraries to deal with them.

Then the data files wouldn't be human readable, making debugging
a lot more hassle.

Cameron Simpson writes:
> There's a common format called Newline Delimited JSON (NDJSON) for just this
> need.
> 
> Just format the outbound records as JSON with no newlines (i.e. make the
> separator a space or the empty string), put a newline at the end of each.
> 
> On ingest, read lines of text, and JSON parse each line separately.

I'll second that. It's very easy to deal with. A shorter name for it
is JSONL -- I use .jsonl for filenames.
https://en.wikipedia.org/wiki/JSON_streaming#Line-delimited_JSON
discusses that as well as several other solutions.

        ...Akkana


From fergalbell at gmail.com  Sun Dec  2 02:00:59 2018
From: fergalbell at gmail.com (fergalbell Bell)
Date: Sun, 2 Dec 2018 07:00:59 +0000
Subject: Unrecognised Arguments
Message-ID: <5c03832b.1c69fb81.cbc82.8981@mx.google.com>

Dear Sir/Madam,

I have been running twitterscraper yesterday with no real problems encountered. I came home and tried to run a twitterscraper command in cmd ? without success.

It said the following for these arguments: -bd 2015-01-01 -ed 2016-01-01 ?lang en unrecognised.

Regards,

Fergal Bell


Sent from Mail for Windows 10



From morphex at gmail.com  Sun Dec  2 06:36:17 2018
From: morphex at gmail.com (Morten W. Petersen)
Date: Sun, 2 Dec 2018 12:36:17 +0100
Subject: ValueError vs IndexError, unpacking arguments with string.split
In-Reply-To: <CAPTjJmrLrH4ODYLg2+TDqEfVmWnnCxpDwxVrFFFLOBv24qU3-g@mail.gmail.com>
References: <CAEtLV=qGJSZ8z+Z7Wqtnp0ckdCvhK_LNdYbcgOVgso_cvM5VXQ@mail.gmail.com>
 <e22b2254-5f88-3ab3-9663-a5abf1826cf4@potatochowder.com>
 <CAEtLV=rfpTYYKXH0A0+DyEOwSQN8Xmo3kU4oTOGCnUwx3xfMBg@mail.gmail.com>
 <792be86a-d5bc-c66d-393e-68ef9fbfe4e0@potatochowder.com>
 <CAEtLV=r62+MduJdzDp-HHbqAgOdB91r2PMF2XHDj4rPrKOjtxw@mail.gmail.com>
 <d8bf19c8-8f2f-0a76-ddcf-bb292ad603d5@potatochowder.com>
 <CAEtLV=qATV9xX1qW2u0g2deTE04nBT35LBODpp4ENeniM08_og@mail.gmail.com>
 <CAPTjJmrAsO8fsNU86pHWXyCT4eXBhRiGDwho3NgZTNSta1y1Dw@mail.gmail.com>
 <CAEtLV=pE2-qVqh6rw3hAquaBDv=BL1xuBSUp=UgjjjwEwB3+FQ@mail.gmail.com>
 <CAPTjJmp9tbt-6nnR=FErQupPEWv12F3B=yhz+MuuV56Tkhnh2g@mail.gmail.com>
 <CAEtLV=peq6aSTwnPGk6HJcstusoi+=0p5zJ56Y+SOW_tHkCzWA@mail.gmail.com>
 <CAPTjJmrLrH4ODYLg2+TDqEfVmWnnCxpDwxVrFFFLOBv24qU3-g@mail.gmail.com>
Message-ID: <CAEtLV=qkG=tu32Hu2DHNEDCT6Sw9fo2mPJM6+_xpE-4hYvwzpg@mail.gmail.com>

On Sun, Dec 2, 2018 at 2:23 AM Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Dec 2, 2018 at 11:55 AM Morten W. Petersen <morphex at gmail.com>
> wrote:
> >
> > On Sat, Dec 1, 2018 at 1:11 AM Chris Angelico <rosuav at gmail.com> wrote:
> >>
> >> On Sat, Dec 1, 2018 at 10:55 AM Morten W. Petersen <morphex at gmail.com>
> wrote:
> >> > But this raises the question of how to write Python code,
> >> > short and sweet, that could handle infinite iterators in
> >> > such an unpack with multiple variables to assign to.
> >> >
> >> > Which I guess is mostly theoretical, as there are other
> >> > ways of designing code to avoid needing to unpack
> >> > an infinite iterator using the * prefix.
> >>
> >> It could only be done with the cooperation of the iterable in
> >> question. For instance, a range object could implement a "remove
> >> first" operation that does this, and several itertools types wouldn't
> >> need to change at all. But it can't be done generically other than the
> >> way it now is (pump the iterator the rest of the way).
> >
> > I wasn't able to follow this, could you elaborate?
> >
>
> The way *x works in unpacking is that the entire iterable gets
> unpacked, and everything gets put into a list that is then assigned to
> x. This is generic, works on any iterable, but doesn't take advantage
> of anything. Consider:
>
> >>> x = range(3, 20, 5)
> >>> first, *rest = x
> >>> first
> 3
> >>> rest
> [8, 13, 18]
>
> If a range object were asked to yield its first-and-rest in this way,
> it could instead return range(8, 20, 5) - add the step onto the start,
> job done. Same if you ask for some off the beginning and some off the
> end And with a number of the itertools iterables/iterators, the "rest"
> wouldn't actually need to change anything, since the iterator will get
> consumed. This would need explicit support from the iterable, though,
> as Python can't know how to do this generically; so there would need
> to be a protocol for "unpack".
>

Aha, yes - I see.

While we're on the subject, I did a test in my Python interpreter:

Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> range(0,3,100)
range(0, 3, 100)
>>> len(range(0,3,100))
1
>>> range(0,100)
range(0, 100)
>>> len(range(0,100))
100
>>>

Where a range with a step, gives the length 1, while a plain range gives
the right length.

I think that's confusing and inconsistent, and it would be nice to have some
"value to be calculated" for the length integer as well.

-Morten

-- 
Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue
Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen
Also playing music and podcasting here:
http://www.mixcloud.com/morten-w-petersen/
On Google+ here https://plus.google.com/107781930037068750156
On Instagram at https://instagram.com/morphexx/


From rosuav at gmail.com  Sun Dec  2 06:46:29 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 2 Dec 2018 22:46:29 +1100
Subject: ValueError vs IndexError, unpacking arguments with string.split
In-Reply-To: <CAEtLV=qkG=tu32Hu2DHNEDCT6Sw9fo2mPJM6+_xpE-4hYvwzpg@mail.gmail.com>
References: <CAEtLV=qGJSZ8z+Z7Wqtnp0ckdCvhK_LNdYbcgOVgso_cvM5VXQ@mail.gmail.com>
 <e22b2254-5f88-3ab3-9663-a5abf1826cf4@potatochowder.com>
 <CAEtLV=rfpTYYKXH0A0+DyEOwSQN8Xmo3kU4oTOGCnUwx3xfMBg@mail.gmail.com>
 <792be86a-d5bc-c66d-393e-68ef9fbfe4e0@potatochowder.com>
 <CAEtLV=r62+MduJdzDp-HHbqAgOdB91r2PMF2XHDj4rPrKOjtxw@mail.gmail.com>
 <d8bf19c8-8f2f-0a76-ddcf-bb292ad603d5@potatochowder.com>
 <CAEtLV=qATV9xX1qW2u0g2deTE04nBT35LBODpp4ENeniM08_og@mail.gmail.com>
 <CAPTjJmrAsO8fsNU86pHWXyCT4eXBhRiGDwho3NgZTNSta1y1Dw@mail.gmail.com>
 <CAEtLV=pE2-qVqh6rw3hAquaBDv=BL1xuBSUp=UgjjjwEwB3+FQ@mail.gmail.com>
 <CAPTjJmp9tbt-6nnR=FErQupPEWv12F3B=yhz+MuuV56Tkhnh2g@mail.gmail.com>
 <CAEtLV=peq6aSTwnPGk6HJcstusoi+=0p5zJ56Y+SOW_tHkCzWA@mail.gmail.com>
 <CAPTjJmrLrH4ODYLg2+TDqEfVmWnnCxpDwxVrFFFLOBv24qU3-g@mail.gmail.com>
 <CAEtLV=qkG=tu32Hu2DHNEDCT6Sw9fo2mPJM6+_xpE-4hYvwzpg@mail.gmail.com>
Message-ID: <CAPTjJmp3tfx-GW9nE1=L3ht7WR6Z-LtaRSnRA93b+MR=r_QOwg@mail.gmail.com>

On Sun, Dec 2, 2018 at 10:36 PM Morten W. Petersen <morphex at gmail.com> wrote:
> While we're on the subject, I did a test in my Python interpreter:
>
> Python 3.6.7 (default, Oct 22 2018, 11:32:17)
> [GCC 8.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> range(0,3,100)
> range(0, 3, 100)
> >>> len(range(0,3,100))
> 1
> >>> range(0,100)
> range(0, 100)
> >>> len(range(0,100))
> 100
> >>>
>
> Where a range with a step, gives the length 1, while a plain range gives
> the right length.
>
> I think that's confusing and inconsistent, and it would be nice to have some
> "value to be calculated" for the length integer as well.
>

Possibly you're misinterpreting the arguments here.

>>> list(range(0, 3, 100))
[0]

class range(object)
 |  range(stop) -> range object
 |  range(start, stop[, step]) -> range object
 |

The *third* argument is the step, so if you were expecting "every
third number up to 100", you'd be looking for this:

>>> len(range(0, 100, 3))
34
>>> len(list(range(0, 100, 3)))
34

To my knowledge, len(x) == len(list(x)) for any core data type that
has a length.

ChrisA


From morphex at gmail.com  Sun Dec  2 07:08:42 2018
From: morphex at gmail.com (Morten W. Petersen)
Date: Sun, 2 Dec 2018 13:08:42 +0100
Subject: ValueError vs IndexError, unpacking arguments with string.split
In-Reply-To: <CAPTjJmp3tfx-GW9nE1=L3ht7WR6Z-LtaRSnRA93b+MR=r_QOwg@mail.gmail.com>
References: <CAEtLV=qGJSZ8z+Z7Wqtnp0ckdCvhK_LNdYbcgOVgso_cvM5VXQ@mail.gmail.com>
 <e22b2254-5f88-3ab3-9663-a5abf1826cf4@potatochowder.com>
 <CAEtLV=rfpTYYKXH0A0+DyEOwSQN8Xmo3kU4oTOGCnUwx3xfMBg@mail.gmail.com>
 <792be86a-d5bc-c66d-393e-68ef9fbfe4e0@potatochowder.com>
 <CAEtLV=r62+MduJdzDp-HHbqAgOdB91r2PMF2XHDj4rPrKOjtxw@mail.gmail.com>
 <d8bf19c8-8f2f-0a76-ddcf-bb292ad603d5@potatochowder.com>
 <CAEtLV=qATV9xX1qW2u0g2deTE04nBT35LBODpp4ENeniM08_og@mail.gmail.com>
 <CAPTjJmrAsO8fsNU86pHWXyCT4eXBhRiGDwho3NgZTNSta1y1Dw@mail.gmail.com>
 <CAEtLV=pE2-qVqh6rw3hAquaBDv=BL1xuBSUp=UgjjjwEwB3+FQ@mail.gmail.com>
 <CAPTjJmp9tbt-6nnR=FErQupPEWv12F3B=yhz+MuuV56Tkhnh2g@mail.gmail.com>
 <CAEtLV=peq6aSTwnPGk6HJcstusoi+=0p5zJ56Y+SOW_tHkCzWA@mail.gmail.com>
 <CAPTjJmrLrH4ODYLg2+TDqEfVmWnnCxpDwxVrFFFLOBv24qU3-g@mail.gmail.com>
 <CAEtLV=qkG=tu32Hu2DHNEDCT6Sw9fo2mPJM6+_xpE-4hYvwzpg@mail.gmail.com>
 <CAPTjJmp3tfx-GW9nE1=L3ht7WR6Z-LtaRSnRA93b+MR=r_QOwg@mail.gmail.com>
Message-ID: <CAEtLV=ob0QkfeVzepSj92Z=SuFRcwW+SG9FwmHTGttuDpcK-4w@mail.gmail.com>

On Sun, Dec 2, 2018 at 12:49 PM Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Dec 2, 2018 at 10:36 PM Morten W. Petersen <morphex at gmail.com>
> wrote:
> > While we're on the subject, I did a test in my Python interpreter:
> >
> > Python 3.6.7 (default, Oct 22 2018, 11:32:17)
> > [GCC 8.2.0] on linux
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> range(0,3,100)
> > range(0, 3, 100)
> > >>> len(range(0,3,100))
> > 1
> > >>> range(0,100)
> > range(0, 100)
> > >>> len(range(0,100))
> > 100
> > >>>
> >
> > Where a range with a step, gives the length 1, while a plain range gives
> > the right length.
> >
> > I think that's confusing and inconsistent, and it would be nice to have
> some
> > "value to be calculated" for the length integer as well.
> >
>
> Possibly you're misinterpreting the arguments here.
>
> >>> list(range(0, 3, 100))
> [0]
>
> class range(object)
>  |  range(stop) -> range object
>  |  range(start, stop[, step]) -> range object
>  |
>
> The *third* argument is the step, so if you were expecting "every
> third number up to 100", you'd be looking for this:
>
> >>> len(range(0, 100, 3))
> 34
> >>> len(list(range(0, 100, 3)))
> 34
>
> To my knowledge, len(x) == len(list(x)) for any core data type that
> has a length.
>

Ah yes, of course.  I haven't had my coffee yet. :)

>>> len(range(0,100,3))
34
>>> range(0,100,3).__len__
<method-wrapper '__len__' of range object at 0x7f144d5e3210>
>>> range(0,100,3).__len__()
34
>>>

But again, I guess an integer-to-be-calculated would be suitable
for iterators where it is very expensive to do a list or other
similar calculation.

These operations take quite a bit of time:

>>> x=list(range(0,100000000,3))
>>> y=len(list(range(0,100000000,3)))
>>>

-Morten

-- 
Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue
Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen
Also playing music and podcasting here:
http://www.mixcloud.com/morten-w-petersen/
On Google+ here https://plus.google.com/107781930037068750156
On Instagram at https://instagram.com/morphexx/


From rosuav at gmail.com  Sun Dec  2 07:17:26 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 2 Dec 2018 23:17:26 +1100
Subject: ValueError vs IndexError, unpacking arguments with string.split
In-Reply-To: <CAEtLV=ob0QkfeVzepSj92Z=SuFRcwW+SG9FwmHTGttuDpcK-4w@mail.gmail.com>
References: <CAEtLV=qGJSZ8z+Z7Wqtnp0ckdCvhK_LNdYbcgOVgso_cvM5VXQ@mail.gmail.com>
 <e22b2254-5f88-3ab3-9663-a5abf1826cf4@potatochowder.com>
 <CAEtLV=rfpTYYKXH0A0+DyEOwSQN8Xmo3kU4oTOGCnUwx3xfMBg@mail.gmail.com>
 <792be86a-d5bc-c66d-393e-68ef9fbfe4e0@potatochowder.com>
 <CAEtLV=r62+MduJdzDp-HHbqAgOdB91r2PMF2XHDj4rPrKOjtxw@mail.gmail.com>
 <d8bf19c8-8f2f-0a76-ddcf-bb292ad603d5@potatochowder.com>
 <CAEtLV=qATV9xX1qW2u0g2deTE04nBT35LBODpp4ENeniM08_og@mail.gmail.com>
 <CAPTjJmrAsO8fsNU86pHWXyCT4eXBhRiGDwho3NgZTNSta1y1Dw@mail.gmail.com>
 <CAEtLV=pE2-qVqh6rw3hAquaBDv=BL1xuBSUp=UgjjjwEwB3+FQ@mail.gmail.com>
 <CAPTjJmp9tbt-6nnR=FErQupPEWv12F3B=yhz+MuuV56Tkhnh2g@mail.gmail.com>
 <CAEtLV=peq6aSTwnPGk6HJcstusoi+=0p5zJ56Y+SOW_tHkCzWA@mail.gmail.com>
 <CAPTjJmrLrH4ODYLg2+TDqEfVmWnnCxpDwxVrFFFLOBv24qU3-g@mail.gmail.com>
 <CAEtLV=qkG=tu32Hu2DHNEDCT6Sw9fo2mPJM6+_xpE-4hYvwzpg@mail.gmail.com>
 <CAPTjJmp3tfx-GW9nE1=L3ht7WR6Z-LtaRSnRA93b+MR=r_QOwg@mail.gmail.com>
 <CAEtLV=ob0QkfeVzepSj92Z=SuFRcwW+SG9FwmHTGttuDpcK-4w@mail.gmail.com>
Message-ID: <CAPTjJmoUzgTBiYPrirSbvGVD23_xoGHw1Xi6s9YV0OdyS48hXA@mail.gmail.com>

On Sun, Dec 2, 2018 at 11:08 PM Morten W. Petersen <morphex at gmail.com> wrote:
>
> On Sun, Dec 2, 2018 at 12:49 PM Chris Angelico <rosuav at gmail.com> wrote:
>> To my knowledge, len(x) == len(list(x)) for any core data type that
>> has a length.
>
> >>> len(range(0,100,3))
> 34
> >>> range(0,100,3).__len__
> <method-wrapper '__len__' of range object at 0x7f144d5e3210>
> >>> range(0,100,3).__len__()
> 34
> >>>
>
> But again, I guess an integer-to-be-calculated would be suitable
> for iterators where it is very expensive to do a list or other
> similar calculation.
>
> These operations take quite a bit of time:
>
> >>> x=list(range(0,100000000,3))
> >>> y=len(list(range(0,100000000,3)))
> >>>
>

Of course they take a lot of time - you're asking for the generation
of some thirty-three million integer objects, each one individually
built and refcounted (in CPython), just so you can count them. But my
point about the equivalency is that, rather than calculating
len(list(range(...))), you can calculate len(range(...)), and you can
be confident that you WILL get the same result.

ChrisA


From songbird at anthive.com  Sun Dec  2 12:40:44 2018
From: songbird at anthive.com (songbird)
Date: Sun, 2 Dec 2018 12:40:44 -0500
Subject: What Python books to you recommend to beginners?
References: <CANc-5UzzRgKHyD5X4VHyOSRfupu4383ug=gHBn55+=wDQd9sHQ@mail.gmail.com>
 <mailman.158.1543416329.2762.python-list@python.org>
 <tutorial-20181128170025@ram.dialup.fu-berlin.de>
 <bVSLD.188159$sH2.175473@fx25.iad>
 <PDF-20181201172846@ram.dialup.fu-berlin.de>
Message-ID: <s1iddf-kb9.ln1@anthive.com>

Stefan Ram wrote:
...

  thank you.  :)

  as references those are useful, but howabout 
something a bit more conceptual or design oriented?

  i have a pretty good idea about various language
features or things to try, but i need a little more
higher level view of how to go about building a
python program.

  i can hack stuff together and it works, but it
doesn't look pretty...  :)


  songbird


From duncan at invalid.invalid  Sun Dec  2 12:56:25 2018
From: duncan at invalid.invalid (duncan smith)
Date: Sun, 2 Dec 2018 17:56:25 +0000
Subject: Injecting methods into instance / class
Message-ID: <e1VMD.1522400$4S3.738114@fx47.iad>

Hello,
      I have a lot of functions that take an instance of a particular
class as the first argument. I want to create corresponding methods in
the class. I have tried the following, which (when called from __init__)
creates the relevant methods in an instance (Python 3.6).


def init_methods(self):
    for func_name, method_name in [('add', '__add__'),
                                   ('subtract', '__sub__')]:
        setattr(self, method_name,
types.MethodType(globals()[func_name], self))


The problem is that e.g.

x.__sub__(y)

works as expected, but

x - y

does not (because special methods are looked up in the class rather than
the instance).

I have tried to find examples of injecting methods into classes without
success. I have tried a few things that intuition suggested might work,
but didn't - like removing the first line above, dedenting and replacing
self with the class. This is only to save typing and make the code
cleaner, but I would like to get it right. Any pointers appreciated. TIA.

Duncan


From morphex at gmail.com  Sun Dec  2 13:15:48 2018
From: morphex at gmail.com (Morten W. Petersen)
Date: Sun, 2 Dec 2018 19:15:48 +0100
Subject: Injecting methods into instance / class
In-Reply-To: <e1VMD.1522400$4S3.738114@fx47.iad>
References: <e1VMD.1522400$4S3.738114@fx47.iad>
Message-ID: <CAEtLV=q7T7-BNxo--Pb7Np7fNK6CmNRkSqbAockSgFK-Wkhj3A@mail.gmail.com>

Hi Duncan.

On Sun, Dec 2, 2018 at 7:02 PM duncan smith <duncan at invalid.invalid> wrote:

> Hello,
>       I have a lot of functions that take an instance of a particular
> class as the first argument. I want to create corresponding methods in
> the class. I have tried the following, which (when called from __init__)
> creates the relevant methods in an instance (Python 3.6).
>
>
> def init_methods(self):
>     for func_name, method_name in [('add', '__add__'),
>                                    ('subtract', '__sub__')]:
>         setattr(self, method_name,
> types.MethodType(globals()[func_name], self))
>
>
> The problem is that e.g.
>
> x.__sub__(y)
>
> works as expected, but
>
> x - y
>
> does not (because special methods are looked up in the class rather than
> the instance).
>
> I have tried to find examples of injecting methods into classes without
> success. I have tried a few things that intuition suggested might work,
> but didn't - like removing the first line above, dedenting and replacing
> self with the class. This is only to save typing and make the code
> cleaner, but I would like to get it right. Any pointers appreciated. TIA.
>

Doesn't this explain things fairly well?

https://docs.python.org/3/reference/datamodel.html#special-lookup

>From what I gather, some methods are always called from the
class object, rather than (instance created from) class object.

-Morten


-- 
Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue
Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen
Also playing music and podcasting here:
http://www.mixcloud.com/morten-w-petersen/
On Google+ here https://plus.google.com/107781930037068750156
On Instagram at https://instagram.com/morphexx/


From morphex at gmail.com  Sun Dec  2 13:35:20 2018
From: morphex at gmail.com (Morten W. Petersen)
Date: Sun, 2 Dec 2018 19:35:20 +0100
Subject: ValueError vs IndexError, unpacking arguments with string.split
In-Reply-To: <CAPTjJmoUzgTBiYPrirSbvGVD23_xoGHw1Xi6s9YV0OdyS48hXA@mail.gmail.com>
References: <CAEtLV=qGJSZ8z+Z7Wqtnp0ckdCvhK_LNdYbcgOVgso_cvM5VXQ@mail.gmail.com>
 <e22b2254-5f88-3ab3-9663-a5abf1826cf4@potatochowder.com>
 <CAEtLV=rfpTYYKXH0A0+DyEOwSQN8Xmo3kU4oTOGCnUwx3xfMBg@mail.gmail.com>
 <792be86a-d5bc-c66d-393e-68ef9fbfe4e0@potatochowder.com>
 <CAEtLV=r62+MduJdzDp-HHbqAgOdB91r2PMF2XHDj4rPrKOjtxw@mail.gmail.com>
 <d8bf19c8-8f2f-0a76-ddcf-bb292ad603d5@potatochowder.com>
 <CAEtLV=qATV9xX1qW2u0g2deTE04nBT35LBODpp4ENeniM08_og@mail.gmail.com>
 <CAPTjJmrAsO8fsNU86pHWXyCT4eXBhRiGDwho3NgZTNSta1y1Dw@mail.gmail.com>
 <CAEtLV=pE2-qVqh6rw3hAquaBDv=BL1xuBSUp=UgjjjwEwB3+FQ@mail.gmail.com>
 <CAPTjJmp9tbt-6nnR=FErQupPEWv12F3B=yhz+MuuV56Tkhnh2g@mail.gmail.com>
 <CAEtLV=peq6aSTwnPGk6HJcstusoi+=0p5zJ56Y+SOW_tHkCzWA@mail.gmail.com>
 <CAPTjJmrLrH4ODYLg2+TDqEfVmWnnCxpDwxVrFFFLOBv24qU3-g@mail.gmail.com>
 <CAEtLV=qkG=tu32Hu2DHNEDCT6Sw9fo2mPJM6+_xpE-4hYvwzpg@mail.gmail.com>
 <CAPTjJmp3tfx-GW9nE1=L3ht7WR6Z-LtaRSnRA93b+MR=r_QOwg@mail.gmail.com>
 <CAEtLV=ob0QkfeVzepSj92Z=SuFRcwW+SG9FwmHTGttuDpcK-4w@mail.gmail.com>
 <CAPTjJmoUzgTBiYPrirSbvGVD23_xoGHw1Xi6s9YV0OdyS48hXA@mail.gmail.com>
Message-ID: <CAEtLV=pQTXMUBqsbKAWOyjoSSxN9t7TEKChaRvPQCbQYZZ1N7g@mail.gmail.com>

On Sun, Dec 2, 2018 at 1:20 PM Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Dec 2, 2018 at 11:08 PM Morten W. Petersen <morphex at gmail.com>
> wrote:
> >
> > On Sun, Dec 2, 2018 at 12:49 PM Chris Angelico <rosuav at gmail.com> wrote:
> >> To my knowledge, len(x) == len(list(x)) for any core data type that
> >> has a length.
> >
> > >>> len(range(0,100,3))
> > 34
> > >>> range(0,100,3).__len__
> > <method-wrapper '__len__' of range object at 0x7f144d5e3210>
> > >>> range(0,100,3).__len__()
> > 34
> > >>>
> >
> > But again, I guess an integer-to-be-calculated would be suitable
> > for iterators where it is very expensive to do a list or other
> > similar calculation.
> >
> > These operations take quite a bit of time:
> >
> > >>> x=list(range(0,100000000,3))
> > >>> y=len(list(range(0,100000000,3)))
> > >>>
> >
>
> Of course they take a lot of time - you're asking for the generation
> of some thirty-three million integer objects, each one individually
> built and refcounted (in CPython), just so you can count them. But my
> point about the equivalency is that, rather than calculating
> len(list(range(...))), you can calculate len(range(...)), and you can
> be confident that you WILL get the same result.
>


Mm, yes, I was aware that it took up references, one more zero
and it would dump a MemoryError.

I hacked a bit of code today, got get my head around the concept
of integer-of-unknown-size, and ended up with this so far:

https://github.com/morphex/misc/blob/98e44915ffeb1cc3ec3050135ea503eb901c01e2/iterator.py

I guess there could be of some (contrived?) use to have an iterator that
will
take different fractions, which will stop iterating over digits once a
repeating
pattern has been found.

Anyway, the more I meddled with the idea of an integer-of-unknown-size,
the more I think that it isn't necessary with the right use of other
concepts
and abstractions.

It could however be a smaller part of a nice, clean pattern/concept.

-Morten


-- 
Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue
Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen
Also playing music and podcasting here:
http://www.mixcloud.com/morten-w-petersen/
On Google+ here https://plus.google.com/107781930037068750156
On Instagram at https://instagram.com/morphexx/


From __peter__ at web.de  Sun Dec  2 13:36:12 2018
From: __peter__ at web.de (Peter Otten)
Date: Sun, 02 Dec 2018 19:36:12 +0100
Subject: Injecting methods into instance / class
References: <e1VMD.1522400$4S3.738114@fx47.iad>
Message-ID: <pu18iq$eiv$1@blaine.gmane.org>

duncan smith wrote:

> Hello,
>       I have a lot of functions that take an instance of a particular
> class as the first argument. I want to create corresponding methods in
> the class. I have tried the following, which (when called from __init__)
> creates the relevant methods in an instance (Python 3.6).
> 
> 
> def init_methods(self):
>     for func_name, method_name in [('add', '__add__'),
>                                    ('subtract', '__sub__')]:
>         setattr(self, method_name,
> types.MethodType(globals()[func_name], self))
> 
> 
> The problem is that e.g.
> 
> x.__sub__(y)
> 
> works as expected, but
> 
> x - y
> 
> does not (because special methods are looked up in the class rather than
> the instance).
> 
> I have tried to find examples of injecting methods into classes without
> success. I have tried a few things that intuition suggested might work,
> but didn't - like removing the first line above, dedenting and replacing
> self with the class. This is only to save typing and make the code
> cleaner, but I would like to get it right. Any pointers appreciated. TIA.

You can do this in the __ init__() method:

$ cat tmp.py
def add(self, other):
    return 42 + other

def init_methods(self):
    cls = type(self)
    for methodname, funcname in [("__add__", "add")]:
        func = globals()[funcname]
        setattr(cls, methodname, func)

class Demo:
    def __init__(self):
        init_methods(self)

x = Demo()
print(x + 100)
$ python3 tmp.py
142

but the __add__() method (to take the example above) is shared by all 
instances. You are either doing too much work by setting it again and again 
in every Demo.__init__() call or you are changing the behaviour of 
previously initialised Demo instances (if the last init_methods() sets the 
method to a different function) and confuse yourself.

Have you considered a mix-in class instead? Like

$ cat tmp.py
def add(self, other):
    return 42 + other

class CommonMethods:
    __add__ = add

class Demo(CommonMethods):
    pass

x = Demo()
print(x + 100)
$ python3 tmp.py
142

This is much cleaner, and if you insist you can set up CommonMethods 
programmatically with

CommonMethods = type(
    "CommonMethods", (),
    {m: globals()[f] for m, f in [("__add__", "add")]}
    # without globals() and probably better: 
    # {m: f for m, f in [("__add__", add)]}
)

though personally I don't see the point.



From songbird at anthive.com  Sun Dec  2 14:43:59 2018
From: songbird at anthive.com (songbird)
Date: Sun, 2 Dec 2018 14:43:59 -0500
Subject: What Python books to you recommend to beginners?
References: <CANc-5UzzRgKHyD5X4VHyOSRfupu4383ug=gHBn55+=wDQd9sHQ@mail.gmail.com>
 <mailman.158.1543416329.2762.python-list@python.org>
 <tutorial-20181128170025@ram.dialup.fu-berlin.de>
 <bVSLD.188159$sH2.175473@fx25.iad>
 <PDF-20181201172846@ram.dialup.fu-berlin.de> <s1iddf-kb9.ln1@anthive.com>
 <38a80epvsi6vseqeeafgq33gboe32danqa@4ax.com>
 <mailman.56.1543777542.25456.python-list@python.org>
Message-ID: <v8pddf-pfa.ln1@anthive.com>

Dennis Lee Bieber wrote:
> On Sun, 2 Dec 2018 12:40:44 -0500, songbird <songbird at anthive.com>
> declaimed the following:
>

>>  as references those are useful, but howabout 
>>something a bit more conceptual or design oriented?
>>
>
> 	At that level, you are not looking for "Python" books but rather
> software engineering texts -- which should provide concepts you then map
> into the language being used.
...

  but i am...


...
> has OOP features -- you don't have to code your own dispatching logic. The
> current favored notation for OOAD modeling is UML, so a book on UML might
> be useful:
>
> Beginning UML 2.0 (O'Reilly)
> UML Distilled (Addison-Wesley)
> Using UML (Addison-Wesley)
>
> 	These are somewhat old and may have newer texts available. It's been
> some 5+ years since I bought any books of this class; my more recent books
> have been microprocessor subjects (Arduino, R-Pi, Beaglebone, PIC and ARM
> Cortex M-series)
...

  thanks,

  i'll look at UML, not sure i want to learn yet another
language on top of python.

  my goal in learning python was to use it as a way of 
picking up OOP concepts in a more concrete way (theory
alone doesn't give me enough hands on the bits i need so
i tend to just do other things instead).

  now that i've used python for a starting project and 
have that project mostly working i want to step back and 
work on the OOP aspects.


  songbird


From duncan at invalid.invalid  Sun Dec  2 14:59:04 2018
From: duncan at invalid.invalid (duncan smith)
Date: Sun, 2 Dec 2018 19:59:04 +0000
Subject: Injecting methods into instance / class
In-Reply-To: <dunder-20181202192605@ram.dialup.fu-berlin.de>
References: <e1VMD.1522400$4S3.738114@fx47.iad>
 <dunder-20181202192605@ram.dialup.fu-berlin.de>
Message-ID: <eQWMD.678455$SN2.557207@fx38.iad>

On 02/12/2018 18:26, Stefan Ram wrote:
> duncan smith <duncan at invalid.invalid> writes:
>> I have tried to find examples of injecting methods into classes without
> 
>   Wouldn't the normal approach be to just define a
>   class with your functions as instance methods?
> 
>   main.py
> 
> class C():
>     def __init__( self, value=0 ):
>         self.value = value
>     def __sub__( self, other ):
>         return C( self.value - other.value )
>     def __str__( self ):
>         return 'C '+ str( self.value )
>     def yourfunctionhere( self ):
>         pass
> 
> c = C(); c.value = 22
> d = C(); d.value = 27
> print( d - c )
> 
>   transcript
> 
> C 5
> 

What I'm trying to avoid is,


    def __sub__(self, other):
        return subtract(self, other)

    def __add__(self, other):
        return add(self, other)

    def __mul__(self, other):
        return multiply(self, other)

    def some_func(self, *args, **kwargs):
        return some_func(self, *args, **kwargs)


for many existing functions. Injecting them as instance methods was
probably not ideal, but I could get it working (apart from the special
methods). Ideally I'd like to take all the functions defined in a
separate module, create ordinary class methods from most of them (with
the same name), then create special methods from the remaining
functions. Cheers.

Duncan


From robertvstepp at gmail.com  Sun Dec  2 16:45:51 2018
From: robertvstepp at gmail.com (boB Stepp)
Date: Sun, 2 Dec 2018 15:45:51 -0600
Subject: What Python books to you recommend to beginners?
In-Reply-To: <v8pddf-pfa.ln1@anthive.com>
References: <CANc-5UzzRgKHyD5X4VHyOSRfupu4383ug=gHBn55+=wDQd9sHQ@mail.gmail.com>
 <mailman.158.1543416329.2762.python-list@python.org>
 <tutorial-20181128170025@ram.dialup.fu-berlin.de>
 <bVSLD.188159$sH2.175473@fx25.iad>
 <PDF-20181201172846@ram.dialup.fu-berlin.de>
 <s1iddf-kb9.ln1@anthive.com> <38a80epvsi6vseqeeafgq33gboe32danqa@4ax.com>
 <mailman.56.1543777542.25456.python-list@python.org>
 <v8pddf-pfa.ln1@anthive.com>
Message-ID: <CANDiX9JOmoi8BHZ1rQuZ-TPJoH1BZoJq88RHByvNHrUhj=AdiQ@mail.gmail.com>

On Sun, Dec 2, 2018 at 1:51 PM songbird <songbird at anthive.com> wrote:
>   my goal in learning python was to use it as a way of
> picking up OOP concepts in a more concrete way (theory
> alone doesn't give me enough hands on the bits i need so
> i tend to just do other things instead).
>
>   now that i've used python for a starting project and
> have that project mostly working i want to step back and
> work on the OOP aspects.

You might want to look at "Python 3 -- Object Oriented Programming" by
Dusty Phillips.  It is copyright 2010, so it won't have the latest,
greatest Python 3 features, but the book's entire focus is teaching
OOP in a Python 3 context.

-- 
boB


From tjol at tjol.eu  Sun Dec  2 18:41:16 2018
From: tjol at tjol.eu (Thomas Jollans)
Date: Mon, 3 Dec 2018 00:41:16 +0100
Subject: Injecting methods into instance / class
In-Reply-To: <e1VMD.1522400$4S3.738114@fx47.iad>
References: <e1VMD.1522400$4S3.738114@fx47.iad>
Message-ID: <ebfcd079-a4a1-c029-9936-bf151b742045@tjol.eu>

On 02/12/2018 18:56, duncan smith wrote:
> Hello,
>       I have a lot of functions that take an instance of a particular
> class as the first argument. I want to create corresponding methods in
> the class. I have tried the following, which (when called from __init__)
> creates the relevant methods in an instance (Python 3.6).

As it sounds like you're creating the classes you want these methods in
yourself, how about using good old-fashioned inheritance instead?


> def init_methods(self):
>     for func_name, method_name in [('add', '__add__'),
>                                    ('subtract', '__sub__')]:
>         setattr(self, method_name,
> types.MethodType(globals()[func_name], self))
> 
> 
> The problem is that e.g.
> 
> x.__sub__(y)
> 
> works as expected, but
> 
> x - y
> 
> does not (because special methods are looked up in the class rather than
> the instance).
> 
> I have tried to find examples of injecting methods into classes without
> success. I have tried a few things that intuition suggested might work,
> but didn't - like removing the first line above, dedenting and replacing
> self with the class. This is only to save typing and make the code
> cleaner, but I would like to get it right. Any pointers appreciated. TIA.
> 
> Duncan
> 



From avigross at verizon.net  Sun Dec  2 19:18:51 2018
From: avigross at verizon.net (Avi Gross)
Date: Sun, 2 Dec 2018 19:18:51 -0500
Subject: dealing with infinite generators
Message-ID: <004101d48a9d$c534afd0$4f9e0f70$@verizon.net>

[SPECULATION ALERT]

I found it interesting as people discussed  how one gets the length of
something set up to follow the iterator protocol and especially anything
that is effectively infinite.

It is possible in python to set a value of "inf" using methods like this:

>>> x = float("inf")
>>> x
inf
>>> x+x
inf
>>> x-x
nan
>>> x*x
inf
>>> x**x
Inf

There is also a "-inf" but length somehow stubbornly seem to remain
non-negative.

So if you made an object that iterates say prime numbers in sequence or
powers of 2 or anything known to be infinite mathematically, if not
practically, and wanted to know the length of the object you could have a
method like this in the class definition:

 def  __len__(self) : return(float("inf"))

There would be no need to count the contents. As stipulated, the count would
take an infinite amount of time, give or take epsilon.

Similarly, if you had some iterator like range(1,1000000,2) that is large
but not infinite, you could set it up so it calculates how many potential
entries it might have when created using simple modular arithmetic, then
save that in a variable. The __next__() method could decrement that variable
and the __len__() method would return that variable. Again, no need to
count.

Now there may well be iterables for which this is not simple or even doable.
If the iterable starts with a list of strings containing all the words in a
book in order and returns one unique word each time by popping the first off
the list then removing any copies, then the length is not known unless you
remove all duplicates at the time you instantiate the instance. But that
sort of reduces it to a list of unique strings and if the point is to quit
iterating long before the list empties, you already did all the work.

Realistically, much python code that looks like:
  for f in iterable:

often does not need to know the length ahead of time.

The other related topic was how to deal with an unpacking like this:

a, b, *c = SOMETHING

If something may be an infinite iterator, or perhaps just a large one, would
you want c instantiated to the rest all at once?

One idea is to not get c back as a list but rather as a (modified) iterator.
This iterator can be expanded or used later in the obvious ways, or simply
ignored.

In newer versions of python the yield statement in a generator can accept a
value from the calling routine. I can imagine having a version of an
iterator that will be called for the first time and return a value, then
some time later, one of the next() calls would include a message back saying
that this call should result in getting back not a value but another
iterator that would continue on. I mean if you asked for:

a, b, *c = range(1,1000, 2)

a would get 1
b would get 3
c would not get  [5, 7, 9 ... 999] as it does now.
c would get something like this:

>>> repr(range(5,1000, 2))
'range(5, 1000, 2)'
>>> type(range(5,1000, 2))
<class 'range'>

Or perhaps get back the results of iter(range(...)) instead.

How would the unpacking operation know what you wanted? My guess is there
are multiple possible ways with anyone complaining no matter what you
choose. But since * and ** are taken to mean unpack into a list or dict, I
wonder if *** is available as in:

a, b, ***c = range(1,1000, 2)

And there are often other solutions possible like making the unpacking
happen with a function call that takes an extra argument as a flag instead
of the above method, or just setting some variable before and/or after that
line to control how greedy the match is, sort of.

Clearly many existing iterators may not be easy to set up this way. The
range() function is particularly easy to modify though as the internals of
the generator function keep track of start, stop, step as shown below:

>>> initial = range(1,10,2)
>>> initial.start, initial.stop, initial.step
(1, 10, 2)

So if asked to return a new iterating object, they can do a computation
internally of what the next result would have been and say it is 7, they
would return range(7,10,2) and presumably exit.

Just some speculation. Adding new features can have odd side effects if you
go to extremes like infinities.




From duncan at invalid.invalid  Sun Dec  2 20:01:58 2018
From: duncan at invalid.invalid (duncan smith)
Date: Mon, 3 Dec 2018 01:01:58 +0000
Subject: Injecting methods into instance / class
In-Reply-To: <mailman.55.1543775783.25456.python-list@python.org>
References: <e1VMD.1522400$4S3.738114@fx47.iad> <pu18iq$eiv$1@blaine.gmane.org>
 <mailman.55.1543775783.25456.python-list@python.org>
Message-ID: <bg%MD.710537$MK2.466644@fx35.iad>

On 02/12/2018 18:36, Peter Otten wrote:
> duncan smith wrote:
> 
>> Hello,
>>       I have a lot of functions that take an instance of a particular
>> class as the first argument. I want to create corresponding methods in
>> the class. I have tried the following, which (when called from __init__)
>> creates the relevant methods in an instance (Python 3.6).
>>
>>
>> def init_methods(self):
>>     for func_name, method_name in [('add', '__add__'),
>>                                    ('subtract', '__sub__')]:
>>         setattr(self, method_name,
>> types.MethodType(globals()[func_name], self))
>>
>>
>> The problem is that e.g.
>>
>> x.__sub__(y)
>>
>> works as expected, but
>>
>> x - y
>>
>> does not (because special methods are looked up in the class rather than
>> the instance).
>>
>> I have tried to find examples of injecting methods into classes without
>> success. I have tried a few things that intuition suggested might work,
>> but didn't - like removing the first line above, dedenting and replacing
>> self with the class. This is only to save typing and make the code
>> cleaner, but I would like to get it right. Any pointers appreciated. TIA.
> 
> You can do this in the __ init__() method:
> 
> $ cat tmp.py
> def add(self, other):
>     return 42 + other
> 
> def init_methods(self):
>     cls = type(self)
>     for methodname, funcname in [("__add__", "add")]:
>         func = globals()[funcname]
>         setattr(cls, methodname, func)
> 
> class Demo:
>     def __init__(self):
>         init_methods(self)
> 
> x = Demo()
> print(x + 100)
> $ python3 tmp.py
> 142
> 
> but the __add__() method (to take the example above) is shared by all 
> instances. You are either doing too much work by setting it again and again 
> in every Demo.__init__() call or you are changing the behaviour of 
> previously initialised Demo instances (if the last init_methods() sets the 
> method to a different function) and confuse yourself.
> 
> Have you considered a mix-in class instead? Like
> 
> $ cat tmp.py
> def add(self, other):
>     return 42 + other
> 
> class CommonMethods:
>     __add__ = add
> 
> class Demo(CommonMethods):
>     pass
> 
> x = Demo()
> print(x + 100)
> $ python3 tmp.py
> 142
> 
> This is much cleaner, and if you insist you can set up CommonMethods 
> programmatically with
> 
> CommonMethods = type(
>     "CommonMethods", (),
>     {m: globals()[f] for m, f in [("__add__", "add")]}
>     # without globals() and probably better: 
>     # {m: f for m, f in [("__add__", add)]}
> )
> 
> though personally I don't see the point.
> 

Ah, I could just bind them within the class,

    mean = mean
    max = max etc.

but I'd somehow convinced myself that didn't work. As the names are the
same I'd probably still prefer to go with something along the lines of

    for name in ['mean', 'max', ...]:
        # create a method

But at least I now have something that works, and I could try something
like your suggestion above to see if I prefer it.

The issue was that some of these "functions" are actually callable class
instances (an attempt to emulate numpy's ufuncs).


class Func:
    def __init__(self, op):
        # op is a numpy ufunc
        self.op = op

    def __call__(self, *args, **kwargs):
        # some logic here to decide
        # which method to call on the
        # basis of the number of arguments
        # and return values

    def one_one(self, x):
        # the method to call for a single argument
        # and single return value


Just rebinding these doesn't work, and these are the callables that need
corresponding special methods. I guess I probably tried binding one of
these first and convinced myself that it didn't work generally.

Many thanks (to everyone who has responded). Not exactly solved, but
I've made some progress. I might just write the methods normally, rather
than looking for a clever shortcut. Cheers.

Duncan


From lutz.horn at outlook.de  Mon Dec  3 01:35:46 2018
From: lutz.horn at outlook.de (Lutz Horn)
Date: Mon, 3 Dec 2018 06:35:46 +0000
Subject: AW: Unrecognised Arguments
In-Reply-To: <5c03832b.1c69fb81.cbc82.8981@mx.google.com>
References: <5c03832b.1c69fb81.cbc82.8981@mx.google.com>
Message-ID: <AM0PR06MB5268AFD692C2C8493BD6E4389EAE0@AM0PR06MB5268.eurprd06.prod.outlook.com>

Hi,

you use the `--lang` option wrongly. Fails:

$ twitterscraper -bd 2015-01-01 -ed 2016-01-01 ?lang en
usage: twitterscraper [-h] [-o OUTPUT] [-l LIMIT] [-a] [-c] [-u] [--lang LANG]
                      [-d] [-bd] [-ed] [-p]
                      query
twitterscraper: error: unrecognized arguments: en

Works:

$ twitterscraper -bd 2015-01-01 -ed 2016-01-01 ?-lang=en

INFO: queries: ['?-lang=en since:2015-01-01 until:2015-01-19', '?-lang=en since:2015-01-19 until:2015-02-06', '?-lang=en since:2015-02-06 until:2015-02-24', '?-lang=en since:2015-02-24 until:2015-03-15', '?-lang=en since:2015-03-15 until:2015-04-02', '?-lang=en since:2015-04-02 until:2015-04-20', '?-lang=en since:2015-04-20 until:2015-05-08', '?-lang=en since:2015-05-08 until:2015-05-27', '?-lang=en since:2015-05-27 until:2015-06-14', '?-lang=en since:2015-06-14 until:2015-07-02', '?-lang=en since:2015-07-02 until:2015-07-20', '?-lang=en since:2015-07-20 until:2015-08-08', '?-lang=en since:2015-08-08 until:2015-08-26', '?-lang=en since:2015-08-26 until:2015-09-13', '?-lang=en since:2015-09-13 until:2015-10-01', '?-lang=en since:2015-10-01 until:2015-10-20', '?-lang=en since:2015-10-20 until:2015-11-07', '?-lang=en since:2015-11-07 until:2015-11-25', '?-lang=en since:2015-11-25 until:2015-12-13', '?-lang=en since:2015-12-13 until:2016-01-01']
INFO: Querying ?-lang=en since:2015-01-01 until:2015-01-19
INFO: Querying ?-lang=en since:2015-01-19 until:2015-02-06
...

Take a look at https://github.com/taspinar/twitterscraper#22-the-cli for details.

Lutz

________________________________________
Von: Python-list <python-list-bounces+lutz.horn=outlook.de at python.org> im Auftrag von fergalbell Bell <fergalbell at gmail.com>
Gesendet: Sonntag, 2. Dezember 2018 08:00
An: python-list at python.org
Betreff: Re: Unrecognised Arguments

Dear Sir/Madam,

I have been running twitterscraper yesterday with no real problems encountered. I came home and tried to run a twitterscraper command in cmd ? without success.

It said the following for these arguments: -bd 2015-01-01 -ed 2016-01-01 ?lang en unrecognised.

Regards,

Fergal Bell


Sent from Mail for Windows 10

--
https://mail.python.org/mailman/listinfo/python-list


From dieter at handshake.de  Mon Dec  3 01:56:14 2018
From: dieter at handshake.de (dieter)
Date: Mon, 03 Dec 2018 07:56:14 +0100
Subject: Unrecognised Arguments
References: <5c03832b.1c69fb81.cbc82.8981@mx.google.com>
Message-ID: <874lbvayk1.fsf@handshake.de>

fergalbell Bell <fergalbell at gmail.com> writes:
> I have been running twitterscraper yesterday with no real problems encountered. I came home and tried to run a twitterscraper command in cmd $(G!9(B without success.
>
> It said the following for these arguments: -bd 2015-01-01 -ed 2016-01-01 $(G!9(Blang en unrecognised.

That we can help you effectively, we usually need the exact call and the error
message -- in possible obtained via cut&paste.

What you provided above might indicate that the program
saw a single argument ("-bd 2015-01-01 -ed 2016-01-01 $(G!9(Blang en")
rather then a sequence of arguments. Potentially because you
have enclosed the arguemnt in quotes?
If this is not the case, come back with the exact command
and error message.



From brunnre8 at gmail.com  Mon Dec  3 02:20:56 2018
From: brunnre8 at gmail.com (brunnre8 at gmail.com)
Date: Mon, 3 Dec 2018 08:20:56 +0100
Subject: Unrecognised Arguments
In-Reply-To: <5c03832b.1c69fb81.cbc82.8981@mx.google.com>
References: <5c03832b.1c69fb81.cbc82.8981@mx.google.com>
Message-ID: <20181203072009.uef4cacnyxgqyim4@feather>

> I have been running twitterscraper yesterday with no real problems encountered. I came home and tried to run a twitterscraper command in cmd ? without success.

Are you running it on the same system as where it was working when you get the error?

> It said the following for these arguments: -bd 2015-01-01 -ed 2016-01-01 ?lang en unrecognised.

Please provide the full command you run as well as the *full* output.

"unrecognised" can mean that you didn't install python or didn't install twitterscraper etc.
Unless you provide the proper information we can only guess.

Greetings,
Reto


From huey.y.jiang at gmail.com  Mon Dec  3 02:58:55 2018
From: huey.y.jiang at gmail.com (huey.y.jiang at gmail.com)
Date: Sun, 2 Dec 2018 23:58:55 -0800 (PST)
Subject: How to run an infinite loop on Menu?
Message-ID: <24dae092-f3b0-47f4-8dc3-8fcc7d701539@googlegroups.com>

Hi Folks,

I need to run an infinite loop on a Menu button, like this:

from Tkinter import *

def run_job():
    i = 0
    while 1:
        i = i + 1
        if i > 100:
            break
     .......

root = Tk()
menu = Menu(root)
root.config(menu=menu)
filemenu = Menu(menu)
menu.add_cascade(label="Jobs", menu=filemenu)
filemenu.add_command(label="Run", command=run_job)
mainloop()


In fact, the def run_job has more things to do. But, it is an infinite loop in essence. It works. However, when the def run_job is running, the menu will be hang there. I have no way to stop it. Even if I raise it with Key-Interupt, the thing was the same: the infinite loop cannot be stopped nicely. The only way was to wait until the Python interpreter notice the problem, and popped out an error message, then I can kill the running program.

Can anybody tell me how to handle this? I wish when variable i reached 100, the run_job will be end, and the Menu "Run" will be free.

Thanks in advance. 



From __peter__ at web.de  Mon Dec  3 03:34:46 2018
From: __peter__ at web.de (Peter Otten)
Date: Mon, 03 Dec 2018 09:34:46 +0100
Subject: Injecting methods into instance / class
References: <e1VMD.1522400$4S3.738114@fx47.iad> <pu18iq$eiv$1@blaine.gmane.org>
 <mailman.55.1543775783.25456.python-list@python.org>
 <bg%MD.710537$MK2.466644@fx35.iad>
Message-ID: <pu2pn6$l3h$1@blaine.gmane.org>

duncan smith wrote:

> On 02/12/2018 18:36, Peter Otten wrote:

>> class CommonMethods:
>>     __add__ = add

> Ah, I could just bind them within the class,
> 
>     mean = mean
>     max = max etc.
> 
> but I'd somehow convinced myself that didn't work. As the names are the
> same I'd probably still prefer to go with something along the lines of
> 
>     for name in ['mean', 'max', ...]:
>         # create a method
> 
> But at least I now have something that works, and I could try something
> like your suggestion above to see if I prefer it.
> 
> The issue was that some of these "functions" are actually callable class
> instances (an attempt to emulate numpy's ufuncs).

For these to work you need to implement the "descriptor protocol", something 
like

$ cat tmp.py
from functools import partial

class Add:
    def __call__(self, left, right):
        return 42 + right

    def __get__(self, inst=None, class_=None):
        if inst is None:
            return self
        return partial(self.__call__, inst)

class Demo:
    __add__ = Add()

x = Demo()
print(x + 100)
$ python3 tmp.py
142




From tjol at tjol.eu  Mon Dec  3 04:03:40 2018
From: tjol at tjol.eu (Thomas Jollans)
Date: Mon, 3 Dec 2018 10:03:40 +0100
Subject: How to run an infinite loop on Menu?
In-Reply-To: <24dae092-f3b0-47f4-8dc3-8fcc7d701539@googlegroups.com>
References: <24dae092-f3b0-47f4-8dc3-8fcc7d701539@googlegroups.com>
Message-ID: <8d38280f-f71e-1576-2650-45280bfa39a3@tjol.eu>

On 03/12/2018 08:58, huey.y.jiang at gmail.com wrote:
> Hi Folks,
> 
> I need to run an infinite loop on a Menu button, like this:
> 
> from Tkinter import *
> 
> def run_job():
>     i = 0
>     while 1:
>         i = i + 1
>         if i > 100:
>             break
>      .......
> 
> root = Tk()
> menu = Menu(root)
> root.config(menu=menu)
> filemenu = Menu(menu)
> menu.add_cascade(label="Jobs", menu=filemenu)
> filemenu.add_command(label="Run", command=run_job)
> mainloop()
> 
> 
> In fact, the def run_job has more things to do. But, it is an infinite loop in essence. It works. However, when the def run_job is running, the menu will be hang there. I have no way to stop it. Even if I raise it with Key-Interupt, the thing was the same: the infinite loop cannot be stopped nicely. The only way was to wait until the Python interpreter notice the problem, and popped out an error message, then I can kill the running program.

Basic rule of thumb for GUI programming: never block the main (GUI)
thread, i.e. never run any long-running task in the main thread.

If you want to start a long-running task from a GUI event, start a
worker thread; from there, you could then perhaps update a progress bar
in your loop. If you want to be able to stop the task from the GUI,
you'll want something like a "stop the task ASAP please" flag/variable
set by the stop button that the loop periodically checks.

> 
> Can anybody tell me how to handle this? I wish when variable i reached 100, the run_job will be end, and the Menu "Run" will be free.
> 
> Thanks in advance. 
> 



From __peter__ at web.de  Mon Dec  3 04:11:41 2018
From: __peter__ at web.de (Peter Otten)
Date: Mon, 03 Dec 2018 10:11:41 +0100
Subject: How to run an infinite loop on Menu?
References: <24dae092-f3b0-47f4-8dc3-8fcc7d701539@googlegroups.com>
Message-ID: <pu2rsc$7n1$1@blaine.gmane.org>

huey.y.jiang at gmail.com wrote:

> Hi Folks,
> 
> I need to run an infinite loop on a Menu button, like this:
> 
> from Tkinter import *
> 
> def run_job():
>     i = 0
>     while 1:
>         i = i + 1
>         if i > 100:
>             break
>      .......
> 
> root = Tk()
> menu = Menu(root)
> root.config(menu=menu)
> filemenu = Menu(menu)
> menu.add_cascade(label="Jobs", menu=filemenu)
> filemenu.add_command(label="Run", command=run_job)
> mainloop()
> 
> 
> In fact, the def run_job has more things to do. But, it is an infinite
> loop in essence. It works. However, when the def run_job is running, the
> menu will be hang there. I have no way to stop it. Even if I raise it with
> Key-Interupt, the thing was the same: the infinite loop cannot be stopped
> nicely. The only way was to wait until the Python interpreter notice the
> problem, and popped out an error message, then I can kill the running
> program.
> 
> Can anybody tell me how to handle this? I wish when variable i reached
> 100, the run_job will be end, and the Menu "Run" will be free.
> 
> Thanks in advance.

You need to break your loop into smaller steps and after each step give the 
user interface a chance to run.

Here's a fancy way using a generator. run_job() implements the steps and 
after each yield next_step() will schedule itself for reinvocation after 100 
milliseconds.

from Tkinter import *

def run_job():
    i = 0
    while True:
        status("i = {}".format(i))
        i = i + 1
        if i > 100:
            break
        yield

def start_job():
    global steps

    status("starting")
    filemenu.entryconfig("Run", state=DISABLED)
    steps = run_job()
    next_step()

def next_step():
    try:
        next(steps)
    except StopIteration:
        status("done")
        filemenu.entryconfig("Run", state=NORMAL)
    else:
        root.after(100, next_step)

def status(message):
    status_label["text"] = message
    print(message)

root = Tk()
menu = Menu(root)
root.config(menu=menu)
root.minsize(100, 50)
filemenu = Menu(menu)
menu.add_cascade(label="Jobs", menu=filemenu)
filemenu.add_command(label="Run", command=start_job)

status_label = Label(root)
status_label.pack()

root.mainloop()




From rhodri at kynesim.co.uk  Mon Dec  3 07:27:32 2018
From: rhodri at kynesim.co.uk (Rhodri James)
Date: Mon, 3 Dec 2018 12:27:32 +0000
Subject: multiple JSON documents in one file, change proposal
In-Reply-To: <87lg59zhed.fsf@elektro.pacujo.net>
References: <87sgzijjjl.fsf@nightsong.com> <87tvjyyyt5.fsf@elektro.pacujo.net>
 <87h8fyjhod.fsf@nightsong.com> <87lg59zhed.fsf@elektro.pacujo.net>
Message-ID: <5c83f8ff-d6f8-d68a-08b5-38ead2a024d1@kynesim.co.uk>

On 01/12/2018 10:10, Marko Rauhamaa wrote:
> In my case, I need an explicit terminator marker to know when a JSON
> value is complete. For example, if I should read from a socket:
> 
>     123

[snip]

You're having to invent a transport protocol to encapsulate your data 
protocol because you don't otherwise have the information you're after 
(packet length in this case).  This shouldn't be a surprise.

The OSI seven-layer model of networking may be overkill for most 
purposes, but it is worth knowing.  We can often get away with omitting 
layers.  You just hit a case where you can't.

-- 
Rhodri James *-* Kynesim Ltd


From songbird at anthive.com  Mon Dec  3 11:14:17 2018
From: songbird at anthive.com (songbird)
Date: Mon, 3 Dec 2018 11:14:17 -0500
Subject: What Python books to you recommend to beginners?
References: <CANc-5UzzRgKHyD5X4VHyOSRfupu4383ug=gHBn55+=wDQd9sHQ@mail.gmail.com>
 <mailman.158.1543416329.2762.python-list@python.org>
 <tutorial-20181128170025@ram.dialup.fu-berlin.de>
 <bVSLD.188159$sH2.175473@fx25.iad>
 <PDF-20181201172846@ram.dialup.fu-berlin.de> <s1iddf-kb9.ln1@anthive.com>
 <38a80epvsi6vseqeeafgq33gboe32danqa@4ax.com>
 <mailman.56.1543777542.25456.python-list@python.org>
 <v8pddf-pfa.ln1@anthive.com>
Message-ID: <pb1gdf-k74.ln1@anthive.com>

A person wrote via e-mail:
> songbird wrote:
>> Dennis Lee Bieber wrote:
...

[post ok'd by them]


> >    my goal in learning python was to use it as a way of
> > picking up OOP concepts in a more concrete way (theory
> > alone doesn't give me enough hands on the bits i need so
> > i tend to just do other things instead).
> > 
> >    now that i've used python for a starting project and
> > have that project mostly working i want to step back and
> > work on the OOP aspects.

> A Scandinavian company put out a good series (I think by Lean Publishing).
> The first is "The Python Apprentice", the next is "The Python Journeyman",
> and presumably there is a third that I have yet to see, ie 'Master'.
> 
> Perhaps "Clean Code in Python", Packt - doesn't match everyone's style.

  will take a look for things and see what i can find.
so i'll put these on the list.

 
> Python's 'batteries included' is exemplified by the Python Standard Library.
> Doug Hellman used to blog a 'Python Module of the Week' (see
> https://pymotw.com/3/) which you might review. He has also published a book
> on the PSL through O'Reilly.

  ok, thanks.  :)

 
> To some extent you may be talking about Design Patterns. Unfortunately, this
> area is terribly confused by certain authors taking concepts from other
> languages, eg Java, and trying to find a Python equivalent. This is a
> grossly non-Pythonic approach. So, I'll join a previous correspondent by
> recommending you stay away from anything 'tainted' by Java-think.

  i would agree with that anyways...


> There are plenty of Python books 'about'. Beyond the 'basics' they tend to
> become more topical, eg Scientific, Financial, Data Analysis... so what
> suits you might not me.

  i'm pretty well read so i can adapt to a lot of
subject matters as long as the concepts are geared
towards what i'm after.  a solid example is a good
thing as long as it covers the concepts - the 
problem i usually have with many examples though is
that they are too simplistic to be interesting enough
but that's just me being picky.  ha...


> Your question is rather broad and open-ended. Do you have access to a decent
> library? What is available there? Have you looked at the various free
> books/downloads and YouTube subscriptions? Happy reading!

  i do have access to a library and it is connected to
the rest of the state and country if i can't get any
of the mentioned items within the network.  though i 
much prefer an electronic copy because the amount of
time it may take me to get through a longer text.


  songbird


From songbird at anthive.com  Mon Dec  3 11:15:21 2018
From: songbird at anthive.com (songbird)
Date: Mon, 3 Dec 2018 11:15:21 -0500
Subject: What Python books to you recommend to beginners?
References: <CANc-5UzzRgKHyD5X4VHyOSRfupu4383ug=gHBn55+=wDQd9sHQ@mail.gmail.com>
 <mailman.158.1543416329.2762.python-list@python.org>
 <tutorial-20181128170025@ram.dialup.fu-berlin.de>
 <bVSLD.188159$sH2.175473@fx25.iad>
 <PDF-20181201172846@ram.dialup.fu-berlin.de> <s1iddf-kb9.ln1@anthive.com>
 <38a80epvsi6vseqeeafgq33gboe32danqa@4ax.com>
 <mailman.56.1543777542.25456.python-list@python.org>
 <v8pddf-pfa.ln1@anthive.com>
 <CANDiX9JOmoi8BHZ1rQuZ-TPJoH1BZoJq88RHByvNHrUhj=AdiQ@mail.gmail.com>
 <mailman.57.1543787166.25456.python-list@python.org>
Message-ID: <pd1gdf-k74.ln1@anthive.com>

boB Stepp wrote:
...
> You might want to look at "Python 3 -- Object Oriented Programming" by
> Dusty Phillips.  It is copyright 2010, so it won't have the latest,
> greatest Python 3 features, but the book's entire focus is teaching
> OOP in a Python 3 context.

  thanks, i'll put it on the list to seek.


  songbird


From avigross at verizon.net  Mon Dec  3 11:41:58 2018
From: avigross at verizon.net (Avi Gross)
Date: Mon, 3 Dec 2018 11:41:58 -0500
Subject: dealing with infinite generators
In-Reply-To: <CAEtLV=omNcFo-wrdN5UaoJLYtGSr=FX3S55_b5Cm3XkgkEDFrA@mail.gmail.com>
References: <004101d48a9d$c534afd0$4f9e0f70$@verizon.net>
 <CAEtLV=omNcFo-wrdN5UaoJLYtGSr=FX3S55_b5Cm3XkgkEDFrA@mail.gmail.com>
Message-ID: <006e01d48b27$1c1153b0$5433fb10$@verizon.net>

Morten,

 

I did not mean that programming languages don?t already have lots of infinities on board. There are plenty of infinite loops that have no terminating conditions and the process continues until it is actively killed. There are recursive function calls that may miss the call to stop because .0000001 is not exactly equal to zero and so on.

 

The issue here is not really new.

 

Any language allows you to create a loop that calls some function repeatedly and will only stop when the function returns nothing or perhaps throws an error.

 

The issue is that python has paradigms that are effectively somewhat hidden loops. A list comprehension is a hidden loop as are its cousins like a dictionary comprehension and of course generators in general. But unlike explicit loops, these constructs may not easily allow you to specify additional code. Yes, you probably can using various techniques. But in real loops you can set flags and increment variables and test for these conditions.

 

I don?t see any easy way to ask that this terminate after a while perhaps by including a modified iterator as the last entry:

 

[ p for p in primes_generator() ]

 

[ p for p in primes_generator() if p < 1000 ]

 

Even if you extended this to check for primes larger than 1,000 it would keep running and just not add additional values. Infinite CPU use and no return to the user of what they asked for.

 

But for the unpacking, example:

 

a, b, c = iterator

 

I do see how it might make sense to use a notation like ***c, as discussed. 

 

Actually, you made me thing about whether the iterator protocol might be amended to deal with infinities.

 

The list comprehension for example takes an ?if? clause. I suspect it could also be expanded to take a ?max? clause after which it stops generating new values. Or, since iterators are objects, you might be able to set a value by setting iterator.max=99 and have the protocol look for and enforce it so you only get the first 99 bottles of beer off the wall.

 

I am curious as to what happens in python if it is trying to expand an iterator and runs out of memory. I can imagine it stopping and throwing an exception but would it unwind whatever it was trying to do or leave a mess?

 

Because python has lots of flexibility and can potentially add any number of dunder hooks to objects, I wonder if there can be a way to designate something as potentially infinite when iterated. Any python facet that implements the iterator protocol might then be warned and choose not to run the protocol an infinite number of times, or perhaps at all. You might have to handle such objects more carefully using iter() and next() yourself in your own carefully crafted loop that does exit in finite time.

 

Just thinking. If more recent versions of python have done something I am not aware of, I would love to know.

 

From: Morten W. Petersen <morphex at gmail.com> 
Sent: Monday, December 3, 2018 7:14 AM
To: avigross at verizon.net
Cc: python-list at python.org
Subject: Re: dealing with infinite generators

 

On Mon, Dec 3, 2018 at 8:11 AM Avi Gross <avigross at verizon.net <mailto:avigross at verizon.net> > wrote:

[SPECULATION ALERT]

I found it interesting as people discussed  how one gets the length of
something set up to follow the iterator protocol and especially anything
that is effectively infinite.

[...]

Just some speculation. Adding new features can have odd side effects if you
go to extremes like infinities.

 

Well I guess one can't argue with that. ?

 

-Morten


-- 

Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue

Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen

Also playing music and podcasting here: http://www.mixcloud.com/morten-w-petersen/

On Google+ here https://plus.google.com/107781930037068750156

On Instagram at https://instagram.com/morphexx/



From grant.b.edwards at gmail.com  Mon Dec  3 12:21:01 2018
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 3 Dec 2018 17:21:01 +0000 (UTC)
Subject: How to reset TCP connection on Linux?
Message-ID: <pu3olt$q14$1@blaine.gmane.org>

How does one reset a TCP connection on Linux?  Note that I want to
reset the connection, not close it.

I need to emulate the rude behavior of a particular app (that's
sending the RST packet) in order to try to find a bug another app (the
one that's receiving the RST).

-- 
Grant Edwards               grant.b.edwards        Yow! Where's th' DAFFY
                                  at               DUCK EXHIBIT??
                              gmail.com            



From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Mon Dec  3 12:39:18 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Mon, 3 Dec 2018 17:39:18 +0000
Subject: conda/anaconda and pip3 (pip)
Message-ID: <pu3po6$2t8$1@gioia.aioe.org>

Hi!

I have an environment created with conda (anaconda3).
There is a package that is unavailable in conda.
Installing it with pip3, with conda env activated, the installation goes
to .local/bin and .local/lib in my home dir (BTW I'm running linux
kubuntu 18.04).
This also has a bad side effect! It reinstalls there some depedencies
already installed in the conda created environment!

Is there a way to avoid this situation?

Thanks for any help.


From jon+usenet at unequivocal.eu  Mon Dec  3 12:53:43 2018
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Mon, 3 Dec 2018 17:53:43 -0000 (UTC)
Subject: How to reset TCP connection on Linux?
References: <pu3olt$q14$1@blaine.gmane.org>
 <mailman.69.1543857804.25456.python-list@python.org>
Message-ID: <slrnq0ard7.339.jon+usenet@sable.unequivocal.eu>

On 2018-12-03, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> How does one reset a TCP connection on Linux?  Note that I want to
> reset the connection, not close it.

Something like the following should work I believe, although I have
not tested it:

    sock.setsockopt(
        socket.SOL_SOCKET,
        socket.SO_LINGER,
        struct.pack('ii', 1, 0)
    )
    sock.close()

Possibly you might want a sock.shutdown(socket.SHUT_RDWR) before
the close().


From grant.b.edwards at gmail.com  Mon Dec  3 14:54:47 2018
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 3 Dec 2018 19:54:47 +0000 (UTC)
Subject: How to reset TCP connection on Linux?
References: <pu3olt$q14$1@blaine.gmane.org>
 <mailman.69.1543857804.25456.python-list@python.org>
 <slrnq0ard7.339.jon+usenet@sable.unequivocal.eu>
Message-ID: <pu41m7$9e$1@blaine.gmane.org>

On 2018-12-03, Jon Ribbens <jon+usenet at unequivocal.eu> wrote:
> On 2018-12-03, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>> How does one reset a TCP connection on Linux?  Note that I want to
>> reset the connection, not close it.
>
> Something like the following should work I believe, although I have
> not tested it:
>
>     sock.setsockopt(
>         socket.SOL_SOCKET,
>         socket.SO_LINGER,
>         struct.pack('ii', 1, 0)
>     )
>     sock.close()

Yes!

Setting SO_LINGER with a 0 timeout is the answer.  I've been googling
for that (on and off) for the past week, and just could not come up
with the right search terms.  Of course adding so_linger to just about
any of my attempts leads immediately to detailed descriptions
including cites to the appropriate section of Stevens and pointers to
where in the kernel it's actually implemented.

Here's a good page:

    http://deepix.github.io/2016/10/21/tcprst.html

-- 
Grant Edwards               grant.b.edwards        Yow! I had a lease on an
                                  at               OEDIPUS COMPLEX back in
                              gmail.com            '81 ...



From PythonList at DancesWithMice.info  Mon Dec  3 15:02:28 2018
From: PythonList at DancesWithMice.info (DL Neil)
Date: Tue, 4 Dec 2018 09:02:28 +1300
Subject: What Python books to you recommend to beginners?
In-Reply-To: <pb1gdf-k74.ln1@anthive.com>
References: <CANc-5UzzRgKHyD5X4VHyOSRfupu4383ug=gHBn55+=wDQd9sHQ@mail.gmail.com>
 <mailman.158.1543416329.2762.python-list@python.org>
 <tutorial-20181128170025@ram.dialup.fu-berlin.de>
 <bVSLD.188159$sH2.175473@fx25.iad>
 <PDF-20181201172846@ram.dialup.fu-berlin.de> <s1iddf-kb9.ln1@anthive.com>
 <38a80epvsi6vseqeeafgq33gboe32danqa@4ax.com>
 <mailman.56.1543777542.25456.python-list@python.org>
 <v8pddf-pfa.ln1@anthive.com> <pb1gdf-k74.ln1@anthive.com>
Message-ID: <723d7386-2f3d-1da5-8daa-efd35221922a@DancesWithMice.info>

Songbird,


> [post ok'd by them]

=this time I've remembered to hit ReplyAll. Duh!


>> There are plenty of Python books 'about'. Beyond the 'basics' they tend to
>> become more topical, eg Scientific, Financial, Data Analysis... so what
>> suits you might not me.
> 
>    i'm pretty well read so i can adapt to a lot of
> subject matters as long as the concepts are geared
> towards what i'm after.  a solid example is a good
> thing as long as it covers the concepts - the
> problem i usually have with many examples though is
> that they are too simplistic to be interesting enough
> but that's just me being picky.  ha...

=yes and no (and nothing to do with you, personally).

=firstly, various specialist areas do tend towards their own 
sub-culture. For (trivial) example, data analysis, statistics, and 
(obviously) pure math guys will tend towards 'algebra', whereas more 
commercial folk preach the virtues of longer, more meaningful object naming.

=secondly, Python has such wide application that there are whole swathes 
of the PSL which people don't use/don't find relevant. Another trivial 
example: 'control' applications and file-based data analysis have little 
use for GUI programming.

=However, some flexibility in your skill set is a strong selling-point; 
so your aim of being "well read" is exactly right (IMHO).


>> Your question is rather broad and open-ended. Do you have access to a decent
>> library? What is available there? Have you looked at the various free
>> books/downloads and YouTube subscriptions? Happy reading!
> 
>    i do have access to a library and it is connected to
> the rest of the state and country if i can't get any
> of the mentioned items within the network.  though i
> much prefer an electronic copy because the amount of
> time it may take me to get through a longer text.

=conversely, I prefer the written (?printed) word for non-fiction and 
reserve eBooks almost entirely for fiction. At first I put that down to 
being a grumpy, old man - we who grew up with books can re-locate 
material by remembering roughly how far 'down' the book, whether on the 
left or right sheet, and perhaps even its proximity to a certain 
diagram/picture. Presumably eBook users (from birth) develop similar, 
yet quite different strategies. In a somewhat ironic twist, recent 
research suggests that even the 'digerati' are finding similarly, and 
asking for 'dead trees' over 'electrons'. So, what would I know?

=Like you, I am blessed with a library system that regards all of its 
libraries as holding a common (and most importantly, user-searchable) 
book stock (for free). There is also a nation-wide library inter-loan 
system (payable). Then there is access to universities' libraries, 
although most would only have borrowing rights at one (involves travel 
in my case). So, when tackling a new topic, I order one or two books, 
then when they arrive, one or two more - thus the later 'latency' period 
is absorbed by my devouring the first set! (and my aberrant greed not 
too obvious - I hope) However, the library idea would work so much 
better if everyone else just left-alone all the books I want, when I 
want them...


=Another thought: it's not a "book" but some/most enjoy Raymond 
Hettinger's talks, available from various PyCon sites/YouTube. He 
specialises in showing the smart techniques that I think you seek/will 
enjoy learning.

=Similarly, in following the various python.org mailing lists I note 
when/where various PyCons will be held around the world - especially 
those too distant to attend. Afterwards I look to see if they've 
recorded the talks. It's a good way for me to become aware of new 
developments in Python itself, and to see the myriad applications and 
(pertinent to your enquiry) the techniques folk use to accomplish such ends.
PS becoming involved in a local PyCon or PUG is a good way to pick-up 
ideas and make contact with folk who have said ideas!

-- 
Regards =dn


From srinivasan.rns at gmail.com  Mon Dec  3 16:16:30 2018
From: srinivasan.rns at gmail.com (srinivasan)
Date: Tue, 4 Dec 2018 02:46:30 +0530
Subject: To check for empty string after a portion of the string in python 3.6
Message-ID: <CAFsTbWdFjdvkW+2Z26OZNo2A9hrQVmPt+4yqssdPfGgqoa3j3Q@mail.gmail.com>

Dear Python Experts,

Could you please help me, as am still learning python syntax, how can
I add conditional check for empty string after running "hcitool scan"
(ie., when there is no Bluetooth devices discovered) ie., after the
word  "Scanning..." , when there are no Bluetooth discover-able
devices and then return False in the below python code snippet?


Command:
~$ sudo hcitool scan
Scanning ...                              -------------------------->
When there are no BT devices
~$ sudo hcitool scan
Scanning ...
64:A2:F9:06:63:79 OnePlus 6 ---------------------------> When there
are BT devices
~$

Python code snippet:
    def bluetooth_scan(self):
        """
            Start bluetooth scanning process.

            :return: Return device info by mac address and name.
        """
        cmd = "sudo hciconfig hci0 up"
        self._helper.execute_cmd_return_code(cmd)

        cmd = "sudo hcitool scan"
        res = self._helper.execute_cmd_output_string(cmd)

        return res

Many Thanks in advance,


From cs at cskk.id.au  Mon Dec  3 16:29:53 2018
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 4 Dec 2018 08:29:53 +1100
Subject: To check for empty string after a portion of the string in
 python 3.6
In-Reply-To: <CAFsTbWdFjdvkW+2Z26OZNo2A9hrQVmPt+4yqssdPfGgqoa3j3Q@mail.gmail.com>
References: <CAFsTbWdFjdvkW+2Z26OZNo2A9hrQVmPt+4yqssdPfGgqoa3j3Q@mail.gmail.com>
Message-ID: <20181203212953.GA63325@cskk.homeip.net>

Note: post returned to the tutor list.

Please DO NOT cross post to multiple lists (i.e. tutor and python-list, 
as you have). This makes things difficult for people who are not on both 
lists. Pick a _single_ list, and use that.

On 04Dec2018 02:46, srinivasan <srinivasan.rns at gmail.com> wrote:
>Could you please help me, as am still learning python syntax, how can
>I add conditional check for empty string after running "hcitool scan"
>(ie., when there is no Bluetooth devices discovered) ie., after the
>word  "Scanning..." , when there are no Bluetooth discover-able
>devices and then return False in the below python code snippet?
>
>
>Command:
>~$ sudo hcitool scan
>Scanning ...                              -------------------------->
>When there are no BT devices
>~$ sudo hcitool scan
>Scanning ...
>64:A2:F9:06:63:79 OnePlus 6 ---------------------------> When there
>are BT devices
>~$
>
>Python code snippet:
>    def bluetooth_scan(self):
>        """
>            Start bluetooth scanning process.
>
>            :return: Return device info by mac address and name.
>        """
>        cmd = "sudo hciconfig hci0 up"
>        self._helper.execute_cmd_return_code(cmd)
>
>        cmd = "sudo hcitool scan"
>        res = self._helper.execute_cmd_output_string(cmd)
>
>        return res

Well you document your function as returning device info by MAC. So part 
the output if "hcitool scan" and for MAC address and name and store that 
in a dictionary (key MAC, value the name). Return the dictionary.

If the dictionary is empty, there were no devices.

Not that like almost all collections, empty dictionaries are "false", so 
you can go:

  bluetooth_devices = scanner.bluetooth_scan()
  if not bluetooth_devices:
    ... no devices found ...

Cheers,
Cameron Simpson <cs at cskk.id.au>


From srinivasan.rns at gmail.com  Mon Dec  3 17:35:11 2018
From: srinivasan.rns at gmail.com (srinivasan)
Date: Tue, 4 Dec 2018 04:05:11 +0530
Subject: To check for empty string after a portion of the string in python
 3.6
In-Reply-To: <20181203212953.GA63325@cskk.homeip.net>
References: <CAFsTbWdFjdvkW+2Z26OZNo2A9hrQVmPt+4yqssdPfGgqoa3j3Q@mail.gmail.com>
 <20181203212953.GA63325@cskk.homeip.net>
Message-ID: <CAFsTbWd6JVDs=nemLMurbMqy5Y-EzC1V9_ZTyf_u1chw3qhX6g@mail.gmail.com>

Dear Mr. Cameron

Thanks a lot for your quick responses, Could you please let me know
when the device is not enabled I get the error " I get the below error
saying "IndexError: list index out of range""

Code snippet:
cmd = "sudo hcitool scan"
res = self._helper.execute_cmd_output_string(cmd)
print("The value of res!!!!", res)
res = self._helper.execute_cmd_output_string(cmd).split("\n", 2)
print("the value!!!!!!!!!!!!", res)
print("the value!!!!!!!!!!!!", res[1])

if __name__ == "__main__":
    m = Bt()
    print(m.bluetooth_scan())


1. Output when device is enabled:
The value of res!!!! Scanning ...
64:A2:F9:06:63:79 OnePlus 6
the value!!!!!!!!!!!! ['Scanning ...', '\t64:A2:F9:06:63:79\tOnePlus 6']
the value!!!!!!!!!!!! 64:A2:F9:06:63:79 OnePlus 6
None

Process finished with exit code 0

2. Output when device is not enabled

When the device is not connected, I get the below error saying
"IndexError: list index out of range"
The value of res!!!! Scanning ...
Traceback (most recent call last):
the value!!!!!!!!!!!! ['Scanning ...']
  File "/home/srinivasan/Downloads/bt_tests/qa/test_library/Bt.py",
line 96, in <module>
    print(m.bluetooth_scan())
  File "/home/srinivasan/Downloads/bt_tests/qa/test_library/Bt.py",
line 74, in bluetooth_scan
    print("the value!!!!!!!!!!!!", res[1])
IndexError: list index out of range

Process finished with exit code 1

Could you please do the needful

Thanks in advance



















On Tue, Dec 4, 2018 at 3:09 AM Cameron Simpson <cs at cskk.id.au> wrote:
>
> Note: post returned to the tutor list.
>
> Please DO NOT cross post to multiple lists (i.e. tutor and python-list,
> as you have). This makes things difficult for people who are not on both
> lists. Pick a _single_ list, and use that.
>
> On 04Dec2018 02:46, srinivasan <srinivasan.rns at gmail.com> wrote:
> >Could you please help me, as am still learning python syntax, how can
> >I add conditional check for empty string after running "hcitool scan"
> >(ie., when there is no Bluetooth devices discovered) ie., after the
> >word  "Scanning..." , when there are no Bluetooth discover-able
> >devices and then return False in the below python code snippet?
> >
> >
> >Command:
> >~$ sudo hcitool scan
> >Scanning ...                              -------------------------->
> >When there are no BT devices
> >~$ sudo hcitool scan
> >Scanning ...
> >64:A2:F9:06:63:79 OnePlus 6 ---------------------------> When there
> >are BT devices
> >~$
> >
> >Python code snippet:
> >    def bluetooth_scan(self):
> >        """
> >            Start bluetooth scanning process.
> >
> >            :return: Return device info by mac address and name.
> >        """
> >        cmd = "sudo hciconfig hci0 up"
> >        self._helper.execute_cmd_return_code(cmd)
> >
> >        cmd = "sudo hcitool scan"
> >        res = self._helper.execute_cmd_output_string(cmd)
> >
> >        return res
>
> Well you document your function as returning device info by MAC. So part
> the output if "hcitool scan" and for MAC address and name and store that
> in a dictionary (key MAC, value the name). Return the dictionary.
>
> If the dictionary is empty, there were no devices.
>
> Not that like almost all collections, empty dictionaries are "false", so
> you can go:
>
>   bluetooth_devices = scanner.bluetooth_scan()
>   if not bluetooth_devices:
>     ... no devices found ...
>
> Cheers,
> Cameron Simpson <cs at cskk.id.au>


From cs at cskk.id.au  Mon Dec  3 18:58:00 2018
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 4 Dec 2018 10:58:00 +1100
Subject: To check for empty string after a portion of the string in
 python 3.6
In-Reply-To: <CAFsTbWd6JVDs=nemLMurbMqy5Y-EzC1V9_ZTyf_u1chw3qhX6g@mail.gmail.com>
References: <CAFsTbWd6JVDs=nemLMurbMqy5Y-EzC1V9_ZTyf_u1chw3qhX6g@mail.gmail.com>
Message-ID: <20181203235800.GA27762@cskk.homeip.net>

Again, _please_ use just one list: tutor or python-list. I've have 
directed replies to the tutor list. - Cameron

On 04Dec2018 04:05, srinivasan <srinivasan.rns at gmail.com> wrote:
>Thanks a lot for your quick responses, Could you please let me know
>when the device is not enabled I get the error " I get the below error
>saying "IndexError: list index out of range""

>Code snippet:
[...]
>res = self._helper.execute_cmd_output_string(cmd)
>print("The value of res!!!!", res)
>res = self._helper.execute_cmd_output_string(cmd).split("\n", 2)
>print("the value!!!!!!!!!!!!", res)
>print("the value!!!!!!!!!!!!", res[1])
>if __name__ == "__main__":
>    m = Bt()
>    print(m.bluetooth_scan())
>
>1. Output when device is enabled:
>The value of res!!!! Scanning ...
>64:A2:F9:06:63:79 OnePlus 6
>the value!!!!!!!!!!!! ['Scanning ...', '\t64:A2:F9:06:63:79\tOnePlus 6']
>the value!!!!!!!!!!!! 64:A2:F9:06:63:79 OnePlus 6
>None

The "None" is because your bluetooth_scan method has no return value (I 
think), which in Python it means it returns None.

>2. Output when device is not enabled
>When the device is not connected, I get the below error saying
>"IndexError: list index out of range"
>The value of res!!!! Scanning ...
>Traceback (most recent call last):
>the value!!!!!!!!!!!! ['Scanning ...']
>  File "/home/srinivasan/Downloads/bt_tests/qa/test_library/Bt.py",
>line 96, in <module>
>    print(m.bluetooth_scan())
>  File "/home/srinivasan/Downloads/bt_tests/qa/test_library/Bt.py",
>line 74, in bluetooth_scan
>    print("the value!!!!!!!!!!!!", res[1])
>IndexError: list index out of range

Well, you've printed your list:

  ['Scanning ...']

It is a single element list. So it has an element at index 0. There is 
no element 1. Therefor Python raises an IndexError if you try to access 
that element.

Cheers,
Cameron Simpson <cs at cskk.id.au>


From oscar.j.benjamin at gmail.com  Tue Dec  4 20:30:24 2018
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 5 Dec 2018 01:30:24 +0000
Subject: Import module from file path
Message-ID: <CAHVvXxTeF+0BM59Zf93o=3vvvZLJqOxY8v+urUE0Fa8vO1Ap7Q@mail.gmail.com>

Hi all,

I'm looking to import a module given a string representing the path to
the .py file defining the module. For example given this setup

mkdir -p a/b/c
touch a/__init__.py
touch a/b/__init__.py
touch a/b/c/__init__.py
touch a/b/c/stuff.py

I have a module a.b.c.stuff which is defined in the file
'/home/oscar/work/project/a/b/c/stuff.py'. Given that a.b.c.stuff is
importable and I have the (relative or absolute) path of stuff.py as a
string I would like to import that module.

I want this to work in 2.7 and 3.4+ and have come up with the
following which works for valid inputs:

import os.path

def import_submodule(filename, rootmodule):
    # Convert from path to module name
    rootdir = os.path.dirname(os.path.dirname(rootmodule.__path__[0]))
    filepath = os.path.relpath(filename, rootdir)
    basename, ext = os.path.splitext(filepath)
    modname = basename.replace('/', '.').replace('\\', '.')
    subattr = modname.split(rootmodule.__name__ + '.')[-1]
    modname = rootmodule.__name__ + '.' + subattr

    # Now import the module
    import importlib
    mod = importlib.import_module(modname)
    return mod

import a
mod = import_submodule('a/b/c/stuff.py', a)
print(dir(mod))

The first part of the above function is the bit that bothers me. I
think there are ways that it could import and run the wrong code if
accidentally given the wrong input (malevolent input is unimportant
here). Also it seems as if there should be a simpler way to get from
the path to the module name...

Cheers,
Oscar


From __peter__ at web.de  Wed Dec  5 02:55:09 2018
From: __peter__ at web.de (Peter Otten)
Date: Wed, 05 Dec 2018 08:55:09 +0100
Subject: Import module from file path
References: <CAHVvXxTeF+0BM59Zf93o=3vvvZLJqOxY8v+urUE0Fa8vO1Ap7Q@mail.gmail.com>
Message-ID: <pu804r$bqs$1@blaine.gmane.org>

Oscar Benjamin wrote:

> Hi all,
> 
> I'm looking to import a module given a string representing the path to
> the .py file defining the module. For example given this setup
> 
> mkdir -p a/b/c
> touch a/__init__.py
> touch a/b/__init__.py
> touch a/b/c/__init__.py
> touch a/b/c/stuff.py
> 
> I have a module a.b.c.stuff which is defined in the file
> '/home/oscar/work/project/a/b/c/stuff.py'. Given that a.b.c.stuff is
> importable and I have the (relative or absolute) path of stuff.py as a
> string I would like to import that module.
> 
> I want this to work in 2.7 and 3.4+ and have come up with the
> following which works for valid inputs:
> 
> import os.path
> 
> def import_submodule(filename, rootmodule):
>     # Convert from path to module name
>     rootdir = os.path.dirname(os.path.dirname(rootmodule.__path__[0]))
>     filepath = os.path.relpath(filename, rootdir)
>     basename, ext = os.path.splitext(filepath)
>     modname = basename.replace('/', '.').replace('\\', '.')
>     subattr = modname.split(rootmodule.__name__ + '.')[-1]
>     modname = rootmodule.__name__ + '.' + subattr
> 
>     # Now import the module
>     import importlib
>     mod = importlib.import_module(modname)
>     return mod
> 
> import a
> mod = import_submodule('a/b/c/stuff.py', a)
> print(dir(mod))
> 
> The first part of the above function is the bit that bothers me. I
> think there are ways that it could import and run the wrong code if
> accidentally given the wrong input (malevolent input is unimportant
> here). Also it seems as if there should be a simpler way to get from
> the path to the module name...

I am not aware of a clean way. I have used


def guess_modulename(filename):
    """Infer module name from filename.

    >>> guess_modulename("/foo/bar/baz.py")
    'baz'
    >>> guess_modulename("/usr/lib/python3.4/logging/handlers.py")
    'logging.handlers'
    """
    if not filename.endswith(".py"):
        raise ValueError("expecting .py file, but got %r" % filename)
    filename = filename[:-3]
    folder, name = os.path.split(filename)
    names = [name]
    while os.path.isfile(os.path.join(folder, "__init__.py")):
        folder, name = os.path.split(folder)
        names.append(name)
    return ".".join(reversed(names))


which unfortunately does not work with namespace packages.



From tjol at tjol.eu  Wed Dec  5 03:51:46 2018
From: tjol at tjol.eu (Thomas Jollans)
Date: Wed, 5 Dec 2018 09:51:46 +0100
Subject: Import module from file path
In-Reply-To: <CAHVvXxTeF+0BM59Zf93o=3vvvZLJqOxY8v+urUE0Fa8vO1Ap7Q@mail.gmail.com>
References: <CAHVvXxTeF+0BM59Zf93o=3vvvZLJqOxY8v+urUE0Fa8vO1Ap7Q@mail.gmail.com>
Message-ID: <d8208cd9-ffff-9808-175f-e71142aa0049@tjol.eu>

On 05/12/2018 02:30, Oscar Benjamin wrote:
> Hi all,
> 
> I'm looking to import a module given a string representing the path to
> the .py file defining the module. For example given this setup
> 
> mkdir -p a/b/c
> touch a/__init__.py
> touch a/b/__init__.py
> touch a/b/c/__init__.py
> touch a/b/c/stuff.py
> 
> I have a module a.b.c.stuff which is defined in the file
> '/home/oscar/work/project/a/b/c/stuff.py'. Given that a.b.c.stuff is
> importable and I have the (relative or absolute) path of stuff.py as a
> string I would like to import that module.
> 
> I want this to work in 2.7 and 3.4+ and have come up with the
> following which works for valid inputs:

I might try something along the lines of: (untested)

if not filename.endswith('.py'):
    raise ValueError('not a .py file')
abs_filename = os.path.abspath(filename)
for path_root in sys.path:
    abs_root = os.path.abspath(path_root)
    if abs_filename.startswith(abs_root):
        rel_filename = os.path.relpath(abs_filename, abs_root)
	if '.' in rel_filename[:-3]:
            # '.' in directory names? can't be right!
            continue
	mod_name = rel_filename[:-3].replace(os.sep, '.')
        try:
            return importlib.import_module(mod_name)
        except ImportError:
            continue
else:
    raise ValueError('not an importable module')

This should work with namespace packages.


> Also it seems as if there should be a simpler way to get from
> the path to the module name...

I doubt it. There's no reason for the import machinery to have such a
mechanism. Besides, not all modules have real file names (e.g. if they
live in zip files)

-- Thomas


From oscar.j.benjamin at gmail.com  Wed Dec  5 06:42:03 2018
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 5 Dec 2018 11:42:03 +0000
Subject: Import module from file path
In-Reply-To: <pu804r$bqs$1@blaine.gmane.org>
References: <CAHVvXxTeF+0BM59Zf93o=3vvvZLJqOxY8v+urUE0Fa8vO1Ap7Q@mail.gmail.com>
 <pu804r$bqs$1@blaine.gmane.org>
Message-ID: <CAHVvXxQDq+5O1a2KCVs4ZPErztSCBeuBVWxMmHSLhHVW8UsSMg@mail.gmail.com>

On Wed, 5 Dec 2018 at 07:57, Peter Otten <__peter__ at web.de> wrote:
>
> Oscar Benjamin wrote:
> >
> > I'm looking to import a module given a string representing the path to
> > the .py file defining the module.
>
> I am not aware of a clean way. I have used
>
> def guess_modulename(filename):
>     """Infer module name from filename.
>
>     >>> guess_modulename("/foo/bar/baz.py")
>     'baz'
>     >>> guess_modulename("/usr/lib/python3.4/logging/handlers.py")
>     'logging.handlers'
>     """
>     if not filename.endswith(".py"):
>         raise ValueError("expecting .py file, but got %r" % filename)
>     filename = filename[:-3]
>     folder, name = os.path.split(filename)
>     names = [name]
>     while os.path.isfile(os.path.join(folder, "__init__.py")):
>         folder, name = os.path.split(folder)
>         names.append(name)
>     return ".".join(reversed(names))
>
>
> which unfortunately does not work with namespace packages.

Thanks Peter. I don't need to worry about namespace packages thankfully.

Thinking about this some more I can see that although the mapping from
module names to file paths needs to be well defined for normal imports
to work the reverse mapping will not always be well defined. For one
there are namespace packages. For another there maybe multiple routes
from sys.path to a particular .py file so that it might be possible to
import a.b.c.stuff as c.stuff if a/b is also in sys.path. Maybe that's
why a ready made solution doesn't seem to exist: the problem itself
isn't well-posed.

--
Oscar


From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Thu Dec  6 02:02:38 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Thu, 6 Dec 2018 07:02:38 +0000
Subject: tkinter resizable text with grid
Message-ID: <puahie$10mt$1@gioia.aioe.org>

Hi!

Does anybody know why this code does not expand the text widget when I
increase the window size (with mouse)? I want height and width but as
minimum (or may be initial) size.

import tkinter as tk

class App:
	def __init__(self,master):
		self.tboard=tk.Text(master,height=40,width=50)
		self.tboard.grid(row=1,column=1,sticky="nsew")
		self.tboard.grid_rowconfigure(1,weight=1)
		self.tboard.grid_columnconfigure(1,weight=1)

root=tk.Tk()
app=App(root)

root.mainloop()

Thanks


From __peter__ at web.de  Thu Dec  6 03:24:01 2018
From: __peter__ at web.de (Peter Otten)
Date: Thu, 06 Dec 2018 09:24:01 +0100
Subject: tkinter resizable text with grid
References: <puahie$10mt$1@gioia.aioe.org>
Message-ID: <puam6v$66b$1@blaine.gmane.org>

Paulo da Silva wrote:

> Does anybody know why this code does not expand the text widget when I
> increase the window size (with mouse)? I want height and width but as
> minimum (or may be initial) size.
> 
> import tkinter as tk
> 
> class App:
>     def __init__(self,master):
>         self.tboard=tk.Text(master,height=40,width=50)
>         self.tboard.grid(row=1,column=1,sticky="nsew")

You have to set the column/row weight of the /master/:

          master.grid_columnconfigure(1, weight=1)
          master.grid_rowconfigure(1, weight=1)

Also, columns and rows usually start with 0.
 
> root=tk.Tk()
> app=App(root)
> 
> root.mainloop()
> 
> Thanks




From marek.mosiewicz at jotel.com.pl  Thu Dec  6 05:48:45 2018
From: marek.mosiewicz at jotel.com.pl (Marek Mosiewicz)
Date: Thu, 06 Dec 2018 11:48:45 +0100
Subject: Creating type evaluation annotation
Message-ID: <1544093325.1893.3.camel@jotel.com.pl>

I'm Java developer,but had some experience with Python based
ERP software. It is quite serious application and I feel 
unconfortable with not having type checking. I do not say language
should be static, but having checking method signature
is big win. For example in refactoring.
I know that Python 3 has possibility to have indicate
type for varibale or param
What could be great to have possibility to annotate any 
class?with what I call "type evaluator" function.
That could check method signature with rules specific?
for given type.
The simplest evaluator would simply check if signature
is same as method definition.?
But it could go futher. It could check whatever is valid?
logic for creating methods in given class and whatever?
are valid params for given method.
Even more powerful evaluator could be not only
valid/not valid indicator. It could validate
and suggest next param or method part name for given
written part of call. That could make python IDEs
much more powerful
>From my experience having signature checking and
hinting params when writing code is really big win.
I could simply refactor 100000 lines of code in
Java, but same for Python is much more difficult

Best regards,
    Marek Mosiewicz




From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Thu Dec  6 09:52:06 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Thu, 6 Dec 2018 14:52:06 +0000
Subject: tkinter resizable text with grid
References: <puahie$10mt$1@gioia.aioe.org> <puam6v$66b$1@blaine.gmane.org>
 <mailman.83.1544084653.25456.python-list@python.org>
Message-ID: <pubd2m$1e0u$1@gioia.aioe.org>

?s 08:24 de 06/12/18, Peter Otten escreveu:
> Paulo da Silva wrote:
> 
...
> 
> You have to set the column/row weight of the /master/:
> 
>           master.grid_columnconfigure(1, weight=1)
>           master.grid_rowconfigure(1, weight=1)
Ok. That works!


> 
> Also, columns and rows usually start with 0.
>  
Yes, I know that. I have other stuff there.

Thank you very much Peter.


From tjol at tjol.eu  Thu Dec  6 13:01:22 2018
From: tjol at tjol.eu (Thomas Jollans)
Date: Thu, 6 Dec 2018 19:01:22 +0100
Subject: Creating type evaluation annotation
In-Reply-To: <1544093325.1893.3.camel@jotel.com.pl>
References: <1544093325.1893.3.camel@jotel.com.pl>
Message-ID: <12781ebd-be03-bd09-6a71-f5aac23648fd@tjol.eu>

On 06/12/2018 11:48, Marek Mosiewicz wrote:
> I'm Java developer,but had some experience with Python based
> ERP software. It is quite serious application and I feel 
> unconfortable with not having type checking. I do not say language
> should be static, but having checking method signature
> is big win. For example in refactoring.

I get the feeling that you're essentially describing mypy.

> I know that Python 3 has possibility to have indicate
> type for varibale or param
> What could be great to have possibility to annotate any 
> class?with what I call "type evaluator" function.
> That could check method signature with rules specific?
> for given type.
> The simplest evaluator would simply check if signature
> is same as method definition.?
> But it could go futher. It could check whatever is valid?
> logic for creating methods in given class and whatever?
> are valid params for given method.
> Even more powerful evaluator could be not only
> valid/not valid indicator. It could validate
> and suggest next param or method part name for given
> written part of call. That could make python IDEs
> much more powerful
> From my experience having signature checking and
> hinting params when writing code is really big win.
> I could simply refactor 100000 lines of code in
> Java, but same for Python is much more difficult
> 
> Best regards,
>     Marek Mosiewicz
> 
> 



From tjreedy at udel.edu  Thu Dec  6 13:05:09 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 6 Dec 2018 13:05:09 -0500
Subject: Creating type evaluation annotation
In-Reply-To: <1544093325.1893.3.camel@jotel.com.pl>
References: <1544093325.1893.3.camel@jotel.com.pl>
Message-ID: <pubo8i$9e9$1@blaine.gmane.org>

On 12/6/2018 5:48 AM, Marek Mosiewicz wrote:
> I'm Java developer,but had some experience with Python based
> ERP software. It is quite serious application and I feel
> unconfortable with not having type checking. I do not say language
> should be static, but having checking method signature
> is big win. For example in refactoring.
> I know that Python 3 has possibility to have indicate
> type for varibale or param
> What could be great to have possibility to annotate any
> class?with what I call "type evaluator" function.
> That could check method signature with rules specific
> for given type.

This and the following needs simple concrete examples.  Withing such, my 
eyes glaze over.

> The simplest evaluator would simply check if signature
> is same as method definition.
> But it could go futher. It could check whatever is valid
> logic for creating methods in given class and whatever
> are valid params for given method.
> Even more powerful evaluator could be not only
> valid/not valid indicator. It could validate
> and suggest next param or method part name for given
> written part of call. That could make python IDEs
> much more powerful
>  From my experience having signature checking and
> hinting params when writing code is really big win.
> I could simply refactor 100000 lines of code in
> Java, but same for Python is much more difficult
> 
> Best regards,
>      Marek Mosiewicz
> 
> 


-- 
Terry Jan Reedy




From jon+usenet at unequivocal.eu  Thu Dec  6 15:06:21 2018
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Thu, 6 Dec 2018 20:06:21 -0000 (UTC)
Subject: Creating type evaluation annotation
References: <1544093325.1893.3.camel@jotel.com.pl>
 <mailman.85.1544116478.25456.python-list@python.org>
Message-ID: <slrnq0j09t.o84.jon+usenet@sable.unequivocal.eu>

On 2018-12-06, Marek Mosiewicz <marek.mosiewicz at jotel.com.pl> wrote:
> I'm Java developer,but had some experience with Python based
> ERP software. It is quite serious application and I feel 
> unconfortable with not having type checking. I do not say language
> should be static, but having checking method signature
> is big win. For example in refactoring.
> I know that Python 3 has possibility to have indicate
> type for varibale or param

Are you talking about http://www.mypy-lang.org/ ?


From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Thu Dec  6 21:00:48 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Fri, 7 Dec 2018 02:00:48 +0000
Subject: tkinter resizable text with grid
References: <puahie$10mt$1@gioia.aioe.org>
 <f6a458c6-5c9d-471a-a9c9-983291980135@googlegroups.com>
Message-ID: <puck8e$t0d$1@gioia.aioe.org>

?s 21:15 de 06/12/18, Rick Johnson escreveu:
> Paulo da Silva wrote:
> 
...

> 
> In Tkinter, if you have a "container"[1] that only has a
> single widget stuffed inside, and, you want that single
> widget to expand to fill the extents of its parent
> container, then, the pack geometry manager is the simplest
> approach.
> 
>     w = create_a_widget()
>     w.pack(fill=X|Y|BOTH, expand=YES)
Yes, I am aware of pack. Unfortunately the code fragment I posted is a
very small part of a larger widget.

...

>     
> I kinda have a love/hate relationship with Tkinter and IDLE.
> On one hand i find them to be practical[2] and simple[3] and
> on the other, i find them to be poorly designed and
> unintuitive. And it's a real shame, because, both of these
> libraries have tons of potential, *IF*, they were designed
> probably and the shortcomings of TclTk were abstracted away
> behind a more Pythonic interface.
> 

I fully agree. Nevertheless, what I miss more is the lack of more
complex mega widgets - scrollable list of widgets with insert, append
and remove methods and perhaps a spreadsheet like widget are two big
ones. There are others smaller, like a single scrollable text with two
scroll bars that hide when not needed, tab multi-choice container, etc ...

Unfortunately I rarely need gui programming and don't have the expertise
to address such task. Being tk so old, I wonder why no one developed
those expansions - continuing tix, for example. There are some
implementations but they seem not being maintained.

Pmw has some of the later, but it is not much stable for python3.

Thanks for responding
Paulo


From dieter at handshake.de  Fri Dec  7 01:55:26 2018
From: dieter at handshake.de (dieter)
Date: Fri, 07 Dec 2018 07:55:26 +0100
Subject: Creating type evaluation annotation
References: <1544093325.1893.3.camel@jotel.com.pl>
Message-ID: <87o99xj069.fsf@handshake.de>

Marek Mosiewicz <marek.mosiewicz at jotel.com.pl> writes:
> ...
> I know that Python 3 has possibility to have indicate
> type for varibale or param
> What could be great to have possibility to annotate any 
> class?with what I call "type evaluator" function.
> That could check method signature with rules specific?
> for given type.
> The simplest evaluator would simply check if signature
> is same as method definition.?
> But it could go futher. It could check whatever is valid?
> logic for creating methods in given class and whatever?
> are valid params for given method.
> Even more powerful evaluator could be not only
> valid/not valid indicator. It could validate
> and suggest next param or method part name for given
> written part of call. That could make python IDEs
> much more powerful
> From my experience having signature checking and
> hinting params when writing code is really big win.

You *can* do all that in Python -- even though you need
something beyond what Python typically comes with.

First, you need a formalisms to describe what should be checked.
There are various alternatives: I am using "zope.interface" (to
describe method signatures) and "zope.schema" (to describe
the "type" of attributes).

Next, you need something that does the checking.
Again, there are alternatives. I prefer to check things
in test suites -- and not permanently during productive use.
But, Python has the powerfull "metaclass" concept.
A metaclass allows you to customize class construction
and e.g. to instrument all defined methods to perform automatic
checks.



From auriocus at gmx.de  Fri Dec  7 02:11:07 2018
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Fri, 7 Dec 2018 08:11:07 +0100
Subject: tkinter resizable text with grid
In-Reply-To: <puck8e$t0d$1@gioia.aioe.org>
References: <puahie$10mt$1@gioia.aioe.org>
 <f6a458c6-5c9d-471a-a9c9-983291980135@googlegroups.com>
 <puck8e$t0d$1@gioia.aioe.org>
Message-ID: <pud6ec$6pc$1@dont-email.me>

Am 07.12.18 um 03:00 schrieb Paulo da Silva:
> ?s 21:15 de 06/12/18, Rick Johnson escreveu:      
>> I kinda have a love/hate relationship with Tkinter and IDLE.
>> On one hand i find them to be practical[2] and simple[3] and
>> on the other, i find them to be poorly designed and
>> unintuitive. And it's a real shame, because, both of these
>> libraries have tons of potential, *IF*, they were designed
>> probably and the shortcomings of TclTk were abstracted away
>> behind a more Pythonic interface.

hahaha...
I'm using Tk from both Tcl and Python, and to me it seems like exactly 
the other way around. Tkinter feels clumsy compared to the "original" 
Tcl/Tk, because of some overzealous OOP-wrapping. "grid" is a good 
example; in Tcl, it is not a member of the widgets but instead a free 
function. It can take more than one slave, so, for instance

	grid .x .y .z
	grid .a  x .b


creates a 3x2 grid of widgets with the lower middle cell empty. No 
fiddling with row and column counting. For some strange reason the 
Tkinter designers decided that grid should be a method of the slave, 
which makes it feel much more clumsy.

> I fully agree. Nevertheless, what I miss more is the lack of more
> complex mega widgets - scrollable list of widgets with insert, append
> and remove methods and perhaps a spreadsheet like widget are two big
> ones. There are others smaller, like a single scrollable text with two
> scroll bars that hide when not needed, tab multi-choice container, etc ...

These widgets do exist for Tcl/Tk. You can use a tablelist for both a 
list of widgets or a multi-column listbox

	https://wiki.tcl-lang.org/page/tablelist

For the spreadsheet, TkTable is another solution which is more close to 
Excel:
	https://wiki.tcl-lang.org/page/Tktable

"Tab multi-choice container" sounds to me like a ttk::notebook widget
	https://wiki.tcl-lang.org/page/notebook

Another handy extension is TkDnD, which brings support for native 
Drag'n'drop, i.e. you can drop files from a file manager into your 
application:

	https://github.com/petasis/tkdnd

- not to be confused with a cheap substitute 
https://kite.com/python/docs/Tkdnd which only supports drag/drop within 
one application instance.

The "problem" with these pacakges, they are written in Tcl and/or C, is 
that they are not distributed along the Tk core and consequently there 
are no well-maintained Python wrappers availabe. In the Tcl community, 
these packages are considered "essential" for more complex GUI 
programming. So instead of complaining about lacking support in Tk, the 
Python community should do their homework and provide wrappers to the 
most common Tk extensions.

	Christian



From mal at europython.eu  Fri Dec  7 12:04:40 2018
From: mal at europython.eu (M.-A. Lemburg)
Date: Fri, 7 Dec 2018 18:04:40 +0100
Subject: EuroPython 2019: Venue and location selected
Message-ID: <1f12293b-ba5d-2999-aa17-8e8afb9ac779@europython.eu>

After a very work intense RFP with more than 40 venues competing, 17
entries, and two rounds of refinements, we are now happy to announce
the winner:

		   EuroPython 2019 will be held in
	      Basel, Switzerland, from July 8 - 14 2019


We will now start work on the contracts and get the organization
going, so that we can all enjoy another edition of EuroPython next
year.


Help spread the word
--------------------

Please help us spread this message by sharing it on your social
networks as widely as possible. Thank you !

Link to the blog post:

https://www.europython-society.org/post/180894308215/europython-2019-venue-and-location-selected

Tweet:

https://twitter.com/europythons/status/1071082681000185857


Thank you,
--
EuroPython Society
https://www.europython-society.org/



From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Fri Dec  7 12:29:09 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Fri, 7 Dec 2018 17:29:09 +0000
Subject: tkinter resizable text with grid
References: <puahie$10mt$1@gioia.aioe.org>
 <f6a458c6-5c9d-471a-a9c9-983291980135@googlegroups.com>
 <puck8e$t0d$1@gioia.aioe.org> <pud6ec$6pc$1@dont-email.me>
Message-ID: <pueal4$1mgh$1@gioia.aioe.org>

?s 07:11 de 07/12/18, Christian Gollwitzer escreveu:
> Am 07.12.18 um 03:00 schrieb Paulo da Silva:
>> ?s 21:15 de 06/12/18, Rick Johnson escreveu:?????
...

> So instead of complaining about lacking support in Tk, the
> Python community should do their homework and provide wrappers to the
> most common Tk extensions.
> 

That was what I did. When I referred tk was in the context of python.
I left tcl/tk long time ago and by that time the problems were the same
as tkinter's today, not to mention the angels sex discussions/wars about
which oop paradigm to use or if use any at all :-)

Regards


From morphex at gmail.com  Fri Dec  7 16:35:39 2018
From: morphex at gmail.com (Morten W. Petersen)
Date: Fri, 7 Dec 2018 22:35:39 +0100
Subject: Focusing on the simple things, KISS, what to use for testing
Message-ID: <CAEtLV=o-E6w7dE5mtKsDSFBdPpw2rAgEX1PspvKSw-cMn_eaqg@mail.gmail.com>

Hi there.

I blogged a bit today, about my surveil project and activity on this
mailing list:

"""
This morning the internet became unavailable, after also being unavailable
this weekend for several days.

So I decided to take a look at my demo board which does surveillance with a
webcam using the surveil app, surveil is here:

https://github.com/morphex/surveil

Well, one thing lead to another (...), and I locked myself out of the demo
board.
"""

The rest of the blog post is here:

http://blogologue.com/blog_entry?id=1544190747X52

Well, I was wondering what framework to use for testing. I see
functional/integration testing as the most useful and powerful thing I can
do, but it is also nice to be able to do unit-testing, and not end up using
different frameworks.

I'm thinking functional testing on the command, through URLs, and through
SMTP/IMAP/POP etc.

I have this old blog software/content management system called the Issue
Dealer, which still runs on Python 2.7 and Zope 2, so it would be nice to
have a testing system which runs on both Python 2 and 3.

This Issue Dealer system is something I might end up ditching, but I guess
it doesn't hurt with some forward thinking with what I choose from now on.

Regards,

Morten

-- 
Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue
Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen
Also playing music and podcasting here:
http://www.mixcloud.com/morten-w-petersen/
On Google+ here https://plus.google.com/107781930037068750156
On Instagram at https://instagram.com/morphexx/


From jfong at ms4.hinet.net  Fri Dec  7 20:17:09 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Fri, 7 Dec 2018 17:17:09 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
Message-ID: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>

>>> 00
0
>>> 03
  File "<stdin>", line 1
    03
     ^
SyntaxError: invalid token
>>>

Any particular reason?


From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Fri Dec  7 20:42:41 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Sat, 8 Dec 2018 01:42:41 +0000
Subject: Why Python don't accept 03 as a number?
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
Message-ID: <puf7if$14e1$1@gioia.aioe.org>

?s 01:17 de 08/12/18, jfong at ms4.hinet.net escreveu:
>>>> 00
> 0
>>>> 03
>   File "<stdin>", line 1
>     03
>      ^
> SyntaxError: invalid token
>>>>
> 
> Any particular reason?
> 

Not sure but I think that after 0 it expects x for hexadecimal, o for
octal, b for binary, ... may be others.

0xa
10

0o10
8

0b10
2


From python at mrabarnett.plus.com  Fri Dec  7 21:04:27 2018
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 8 Dec 2018 02:04:27 +0000
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
Message-ID: <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>

On 2018-12-08 01:17, jfong at ms4.hinet.net wrote:
>>>> 00
> 0
>>>> 03
>    File "<stdin>", line 1
>      03
>       ^
> SyntaxError: invalid token
>>>>
> 
> Any particular reason?
> 
Before Python 3, a leading 0 in an integer literal would indicate an 
octal (base 8) number.

In Python 2.7:
 >>> 010
8

That notation was borrowed from C.

The hexadecimal (base 16) notation of a leading 0x was also borrowed 
from C (it was a later introduction to the language).

Python also has the binary (base 2) notation of a leading 0b.

For Python 3, it was felt that it would be clearer to have an octal 
notation that didn't use just a leading 0, so it was changed to a 
leading 0o.

The old form is now invalid in order to reduce the chance of bugs.

If you're coming from Python 2, you might expect that 010 is still 
octal, and if you're not familiar with that notation, you might expect 
that it's decimal.


From jfong at ms4.hinet.net  Fri Dec  7 21:40:30 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Fri, 7 Dec 2018 18:40:30 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <mailman.91.1544234677.25456.python-list@python.org>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
Message-ID: <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>

MRAB at 2018/12/8 UTC+8 AM10:04:51 wrote:
> Before Python 3, a leading 0 in an integer literal would indicate an 
> octal (base 8) number.

So, the reason is historical.

> The old form is now invalid in order to reduce the chance of bugs.

I encounter this problem on trying to do something like this:
    eval('03 + 00 + 15')
It takes me some efforts to get rid of those leading zeros:-(

Hope someday 03 can be accepted as a valid decimal number in Python 3.

Thank you for explaining.

--Jach




From rosuav at gmail.com  Fri Dec  7 22:13:51 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 8 Dec 2018 14:13:51 +1100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
Message-ID: <CAPTjJmo3aZwYqKzuHCH=K7PFwrk5hiUoOsO2STq5un3jX19sNQ@mail.gmail.com>

On Sat, Dec 8, 2018 at 1:46 PM <jfong at ms4.hinet.net> wrote:
>
> MRAB at 2018/12/8 UTC+8 AM10:04:51 wrote:
> > Before Python 3, a leading 0 in an integer literal would indicate an
> > octal (base 8) number.
>
> So, the reason is historical.
>
> > The old form is now invalid in order to reduce the chance of bugs.
>
> I encounter this problem on trying to do something like this:
>     eval('03 + 00 + 15')
> It takes me some efforts to get rid of those leading zeros:-(
>
> Hope someday 03 can be accepted as a valid decimal number in Python 3.
>

Definitely not. What happens to all the code that used to be legal and
meant octal, and would become legal again but with a different
meaning? It'd be bad enough to have Python interpret something in a
way that's subtly different from the way other languages do (annoying,
but livable), but to do that across versions of the language would be
an incredibly bad idea.

ChrisA


From ian.g.kelly at gmail.com  Fri Dec  7 22:27:41 2018
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Fri, 7 Dec 2018 20:27:41 -0700
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
Message-ID: <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>

On Fri, Dec 7, 2018 at 7:47 PM <jfong at ms4.hinet.net> wrote:
>
> MRAB at 2018/12/8 UTC+8 AM10:04:51 wrote:
> > Before Python 3, a leading 0 in an integer literal would indicate an
> > octal (base 8) number.
>
> So, the reason is historical.
>
> > The old form is now invalid in order to reduce the chance of bugs.
>
> I encounter this problem on trying to do something like this:
>     eval('03 + 00 + 15')
> It takes me some efforts to get rid of those leading zeros:-(

What is it exactly that you're trying to accomplish with this? Perhaps
there's a better way than using eval.


From pfeiffer at cs.nmsu.edu  Fri Dec  7 22:49:45 2018
From: pfeiffer at cs.nmsu.edu (Joe Pfeiffer)
Date: Fri, 07 Dec 2018 20:49:45 -0700
Subject: Why Python don't accept 03 as a number?
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
Message-ID: <1bpnucd6ee.fsf@pfeifferfamily.net>

jfong at ms4.hinet.net writes:

> MRAB at 2018/12/8 UTC+8 AM10:04:51 wrote:
>> Before Python 3, a leading 0 in an integer literal would indicate an 
>> octal (base 8) number.
>
> So, the reason is historical.
>
>> The old form is now invalid in order to reduce the chance of bugs.
>
> I encounter this problem on trying to do something like this:
>     eval('03 + 00 + 15')
> It takes me some efforts to get rid of those leading zeros:-(
>
> Hope someday 03 can be accepted as a valid decimal number in Python 3.
>
> Thank you for explaining.
>
> --Jach

I'd say we *really* don't want that.  We'd have old C programmers (like
me) expecting 010 to mean 8, and getting really confused...


From python at mrabarnett.plus.com  Fri Dec  7 23:17:09 2018
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 8 Dec 2018 04:17:09 +0000
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <1bpnucd6ee.fsf@pfeifferfamily.net>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <1bpnucd6ee.fsf@pfeifferfamily.net>
Message-ID: <38e7bc98-8427-ecba-3fb6-37bb2b230369@mrabarnett.plus.com>

On 2018-12-08 03:49, Joe Pfeiffer wrote:
> jfong at ms4.hinet.net writes:
> 
>> MRAB at 2018/12/8 UTC+8 AM10:04:51 wrote:
>>> Before Python 3, a leading 0 in an integer literal would indicate an 
>>> octal (base 8) number.
>>
>> So, the reason is historical.
>>
>>> The old form is now invalid in order to reduce the chance of bugs.
>>
>> I encounter this problem on trying to do something like this:
>>     eval('03 + 00 + 15')
>> It takes me some efforts to get rid of those leading zeros:-(
>>
>> Hope someday 03 can be accepted as a valid decimal number in Python 3.
>>
>> Thank you for explaining.
>>
>> --Jach
> 
> I'd say we *really* don't want that.  We'd have old C programmers (like
> me) expecting 010 to mean 8, and getting really confused...
> 
We could just wait until all the old C programmers have died. :-)


From jfong at ms4.hinet.net  Fri Dec  7 23:24:45 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Fri, 7 Dec 2018 20:24:45 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <mailman.93.1544239701.25456.python-list@python.org>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
Message-ID: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>

Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
> What is it exactly that you're trying to accomplish with this? Perhaps
> there's a better way than using eval.

This problem comes from solving a word puzzle,
    ab + aa + cd == ce
Each character will be translate to a digit and evaluate the correctness,
    03 + 00 + 15 == 18



From jfong at ms4.hinet.net  Fri Dec  7 23:34:44 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Fri, 7 Dec 2018 20:34:44 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
Message-ID: <33b01fa6-9b23-4fb9-9092-8a9c7a4fc80e@googlegroups.com>

I can understand the difficulty of throwing old thing away and accept new one in human. There seems have a huge inertia there. This phenomenon appears on every aspects, not only on the transition from Python2 to Python3. But, as a new comer of Python like me, I have no difficulty to accept it because of 03 is a valid number in my daily life and never had the experience of treating 010 as 8:-)

MBAB wrote:
> We could just wait until all the old C programmers have died. :-)

Yes, it's the nature way.


From cs at cskk.id.au  Sat Dec  8 00:00:02 2018
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 8 Dec 2018 16:00:02 +1100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
Message-ID: <20181208050002.GA62588@cskk.homeip.net>

On 07Dec2018 20:24, Jach Fong <jfong at ms4.hinet.net> wrote:
>Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
>> What is it exactly that you're trying to accomplish with this? Perhaps
>> there's a better way than using eval.
>
>This problem comes from solving a word puzzle,
>    ab + aa + cd == ce
>Each character will be translate to a digit and evaluate the correctness,
>    03 + 00 + 15 == 18

Then you should be evaluating the digits and assembling values from 
them. Not trying to shoehorn a string through something that _might_ 
accept this string and do what you want. In Python 2 it will accept your 
string and not do what you want; at least in Python 3 it doesn't accept 
your string.

My point here is that the structure of your puzzle doesn't map directly 
into a naive python statement, and you shouldn't be pretending it might.

Cheers,
Cameron Simpson <cs at cskk.id.au>


From avigross at verizon.net  Sat Dec  8 01:08:59 2018
From: avigross at verizon.net (Avi Gross)
Date: Sat, 8 Dec 2018 01:08:59 -0500
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
 <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
Message-ID: <002401d48ebc$82f70690$88e513b0$@verizon.net>

[[READERS DIGEST CONDENSED ANSWER: use int("string") ]]

Since we all agree python will not make notations like "05" work
indefinitely, and the need expressed is how to solve a symbolic puzzle (see
message below) then it makes sense to look at alternate representations.

I have a question first. How are you solving your puzzles? 

    ab + aa + cd == ce

Why does 05 ever even appear in your solution?

Are you generating all possible answers by setting each variable to one of 0
to 9 then the second to any of the remaining nine choices then the third to
the remaining 8 and so on? For any method like that, you can presumably make
each component like

ab = 10*a + b

in the loop.

Similarly for aa and cd and ce. If the equality above is true, you found the
solution and break out. If there can be multiple solutions, note the
solution and keep going. But note for the 5 variables above, you are testing
10*9*8*7*6 combinations.

Another idea is to use strings like "05" as bizarrely, the function int()
seems to be an ex eption that does NOT care about leading whitespace or
zeroes:

>>> int("05")
5
>>> int("  0005")
5

And even handles all zeroes:

>>> int("000000")
0

You can also use lstrip() with an argument to remove zeros:

>>> a = eval("05".lstrip("0"))
	     
>>> a
	     
5

If you are in a situation where you only want to remove leading zeroes if
the following character is a digit and not "o" or "b" or "x", use regular
expressions or other techniques.

I will just toss in the possible use of the SymPy module to do actual
symbolic computations to solve some of these. Perhaps a tad advanced.


-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of jfong at ms4.hinet.net
Sent: Friday, December 7, 2018 11:25 PM
To: python-list at python.org
Subject: Re: Why Python don't accept 03 as a number?

Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
> What is it exactly that you're trying to accomplish with this? Perhaps 
> there's a better way than using eval.

This problem comes from solving a word puzzle,
    ab + aa + cd == ce
Each character will be translate to a digit and evaluate the correctness,
    03 + 00 + 15 == 18

--
https://mail.python.org/mailman/listinfo/python-list



From henrik.bengtsson at gmail.com  Sat Dec  8 01:53:45 2018
From: henrik.bengtsson at gmail.com (Henrik Bengtsson)
Date: Fri, 7 Dec 2018 22:53:45 -0800
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <20181208050002.GA62588@cskk.homeip.net>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
Message-ID: <CAFDcVCRVrEB2Sp2Lhgn9kSUc=LiBw13Vja87H2sDMSCFpAd0dg@mail.gmail.com>

A comment from the sideline: one could imagine extending the Python syntax
with a (optional) 0d prefix that allows for explicit specification of
decimal values. They would "complete" the family:

* 0b: binary number
* 0o: octal number
* 0d: decimal number
* 0x: hexadecimal number

I understand that changing the syntax/parser is a major move. I wouldn't be
surprised if others brought up 0d before.

My $.02

Henrik


On Fri, Dec 7, 2018, 21:12 Cameron Simpson <cs at cskk.id.au wrote:

> On 07Dec2018 20:24, Jach Fong <jfong at ms4.hinet.net> wrote:
> >Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
> >> What is it exactly that you're trying to accomplish with this? Perhaps
> >> there's a better way than using eval.
> >
> >This problem comes from solving a word puzzle,
> >    ab + aa + cd == ce
> >Each character will be translate to a digit and evaluate the correctness,
> >    03 + 00 + 15 == 18
>
> Then you should be evaluating the digits and assembling values from
> them. Not trying to shoehorn a string through something that _might_
> accept this string and do what you want. In Python 2 it will accept your
> string and not do what you want; at least in Python 3 it doesn't accept
> your string.
>
> My point here is that the structure of your puzzle doesn't map directly
> into a naive python statement, and you shouldn't be pretending it might.
>
> Cheers,
> Cameron Simpson <cs at cskk.id.au>
> --
> https://mail.python.org/mailman/listinfo/python-list
>


From storchaka at gmail.com  Sat Dec  8 01:59:01 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 8 Dec 2018 08:59:01 +0200
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
Message-ID: <pufpvh$1pt$1@blaine.gmane.org>

08.12.18 03:17, jfong at ms4.hinet.net ????:
>>>> 00
> 0
>>>> 03
>    File "<stdin>", line 1
>      03
>       ^
> SyntaxError: invalid token


In Python 3.8 the error message will be more informative:

>>> 03
   File "<stdin>", line 1
SyntaxError: leading zeros in decimal integer literals are not 
permitted; use an 0o prefix for octal integers



From ian.g.kelly at gmail.com  Sat Dec  8 03:35:43 2018
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 8 Dec 2018 01:35:43 -0700
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <CAFDcVCRVrEB2Sp2Lhgn9kSUc=LiBw13Vja87H2sDMSCFpAd0dg@mail.gmail.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <CAFDcVCRVrEB2Sp2Lhgn9kSUc=LiBw13Vja87H2sDMSCFpAd0dg@mail.gmail.com>
Message-ID: <CALwzidkekC9UUKdDdFb04UsVubu6udkFGfZXVJS1Q+5eqnj2Cg@mail.gmail.com>

On Fri, Dec 7, 2018 at 11:56 PM Henrik Bengtsson
<henrik.bengtsson at gmail.com> wrote:
>
> A comment from the sideline: one could imagine extending the Python syntax
> with a (optional) 0d prefix that allows for explicit specification of
> decimal values. They would "complete" the family:
>
> * 0b: binary number
> * 0o: octal number
> * 0d: decimal number
> * 0x: hexadecimal number

That's nice and elegant, but what would be the use case?


From storchaka at gmail.com  Sat Dec  8 04:20:11 2018
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 8 Dec 2018 11:20:11 +0200
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <CAFDcVCRVrEB2Sp2Lhgn9kSUc=LiBw13Vja87H2sDMSCFpAd0dg@mail.gmail.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <CAFDcVCRVrEB2Sp2Lhgn9kSUc=LiBw13Vja87H2sDMSCFpAd0dg@mail.gmail.com>
Message-ID: <pug288$66j$1@blaine.gmane.org>

08.12.18 08:53, Henrik Bengtsson ????:
> A comment from the sideline: one could imagine extending the Python syntax
> with a (optional) 0d prefix that allows for explicit specification of
> decimal values. They would "complete" the family:
> 
> * 0b: binary number
> * 0o: octal number
> * 0d: decimal number
> * 0x: hexadecimal number
> 
> I understand that changing the syntax/parser is a major move. I wouldn't be
> surprised if others brought up 0d before.

It is more likely that 'd' will be used for Decimal literals.

1d-2 == Decimal('0.01')



From jfong at ms4.hinet.net  Sat Dec  8 04:26:16 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Sat, 8 Dec 2018 01:26:16 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <mailman.96.1544249346.25456.python-list@python.org>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
 <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <002401d48ebc$82f70690$88e513b0$@verizon.net>
 <mailman.96.1544249346.25456.python-list@python.org>
Message-ID: <f79c5efa-5baf-41ed-92bc-89257a8ecd87@googlegroups.com>

Avi Gross at 2018/12/8 UTC+8 PM2:09:20 wrote:
> [[READERS DIGEST CONDENSED ANSWER: use int("string") ]]
> 
> Since we all agree python will not make notations like "05" work
> indefinitely, and the need expressed is how to solve a symbolic puzzle (see
> message below) then it makes sense to look at alternate representations.
> 
> I have a question first. How are you solving your puzzles? 
> 
>     ab + aa + cd == ce

Try all the combinations:-)

The only way I can think of is try-error. It takes no more 10 lines to go from "ab + aa + cd == ce" to yield one correct answer, such as "03 + 00 + 15 == 18", using itertools' permutations(), string's translate() and re.

> Why does 05 ever even appear in your solution?

I don't know. There is total 192 answers for this puzzle anyway.

> Are you generating all possible answers by setting each variable to one of 0
> to 9 then the second to any of the remaining nine choices then the third to
> the remaining 8 and so on? For any method like that, you can presumably make
> each component like
> 
> ab = 10*a + b
> 
> in the loop.
> 
> Similarly for aa and cd and ce. If the equality above is true, you found the
> solution and break out. If there can be multiple solutions, note the
> solution and keep going. But note for the 5 variables above, you are testing
> 10*9*8*7*6 combinations.
> 
> Another idea is to use strings like "05" as bizarrely, the function int()
> seems to be an ex eption that does NOT care about leading whitespace or
> zeroes:
> 
> >>> int("05")
> 5
> >>> int("  0005")
> 5
> 
> And even handles all zeroes:
> 
> >>> int("000000")
> 0
> 
> You can also use lstrip() with an argument to remove zeros:
> 
> >>> a = eval("05".lstrip("0"))
> 	     
> >>> a
> 	     
> 5
> 
> If you are in a situation where you only want to remove leading zeroes if
> the following character is a digit and not "o" or "b" or "x", use regular
> expressions or other techniques.

As far as the leading zero problem was concerned, the simplest way is using re.sub()

> I will just toss in the possible use of the SymPy module to do actual
> symbolic computations to solve some of these. Perhaps a tad advanced.

I don't know SymPy and if it can shorten the execution time. But I am very curious about if there is other algorithm which can apply to this problem:-)

--Jach


From Richard at Damon-Family.org  Sat Dec  8 08:18:45 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Sat, 8 Dec 2018 08:18:45 -0500
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
 <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
Message-ID: <013c0081-94b1-8441-b71a-f5287a9ad13f@Damon-Family.org>

On 12/7/18 11:24 PM, jfong at ms4.hinet.net wrote:
> Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
>> What is it exactly that you're trying to accomplish with this? Perhaps
>> there's a better way than using eval.
> This problem comes from solving a word puzzle,
>     ab + aa + cd == ce
> Each character will be translate to a digit and evaluate the correctness,
>     03 + 00 + 15 == 18
>
For the classic version of that problem, a = 0 is not allowed, since you
never in 'normal' life write number with leading zeros, the alpha-math
problems never have leading zeros either.

As other have said, the work to compute the value yourself rather than
due the text substitution and eval is close to comparable.

-- 
Richard Damon



From grant.b.edwards at gmail.com  Sat Dec  8 11:49:29 2018
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Sat, 8 Dec 2018 16:49:29 +0000 (UTC)
Subject: Why Python don't accept 03 as a number?
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
Message-ID: <pugsmp$i6a$1@blaine.gmane.org>

On 2018-12-08, Cameron Simpson <cs at cskk.id.au> wrote:
> On 07Dec2018 20:24, Jach Fong <jfong at ms4.hinet.net> wrote:
>>Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
>>> What is it exactly that you're trying to accomplish with this? Perhaps
>>> there's a better way than using eval.
>>
>>This problem comes from solving a word puzzle,
>>    ab + aa + cd == ce
>>Each character will be translate to a digit and evaluate the correctness,
>>    03 + 00 + 15 == 18
>
> Then you should be evaluating the digits and assembling values from 
> them. Not trying to shoehorn a string through something that _might_ 
> accept this string and do what you want. In Python 2 it will accept your 
> string and not do what you want; at least in Python 3 it doesn't accept 
> your string.

Just to be clear: you do _not_ want to use eval on the string.

If you're not the one who created the string, it might wipe your hard
drive or empty your bank account.  If you _are_ the one who created
the string, then generate the desired result instead.

-- 
Grant







From avigross at verizon.net  Sat Dec  8 12:40:06 2018
From: avigross at verizon.net (Avi Gross)
Date: Sat, 8 Dec 2018 12:40:06 -0500
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <f79c5efa-5baf-41ed-92bc-89257a8ecd87@googlegroups.com>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
 <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <002401d48ebc$82f70690$88e513b0$@verizon.net>
 <mailman.96.1544249346.25456.python-list@python.org>
 <f79c5efa-5baf-41ed-92bc-89257a8ecd87@googlegroups.com>
Message-ID: <005701d48f1d$0f253ff0$2d6fbfd0$@verizon.net>

Jach,

Just for fun, I looked at the  puzzle you asked about and solved it several
ways without running into your 03 problem at all. There are more efficient
solutions than total brute force.

Anyone not interested, stop here, please. After my explanations, I show a
python program then the output it produces.

The comments in the python program (below) explain the solution classes but
let me suggest a little algebra simplifies the problem so fewer brute force
iterations are needed.

The simplistic algorithm can be written as a single list comprehension with
(in this case) 5 "for" clauses and one "if" clause.

You are solving for: ab + aa + cd == ce

So 
for all digits possible for a
for all remaining digits possible for b each iteration
for all remaining digits possible for c each iteration
for all remaining digits possible for d each iteration
for all remaining digits possible for e each iteration
if the condition applies.

Clearly the above is deeply nested with 10!/5! Iterations or 30,240.

But a little algebra simplifies the solution so c drops out of the equation
as shown below in the comments.
The test becomes:  21*a + b + d - e == 0

You can apply that as the condition using four loops. You get 32 solutions
and for each you can let c be any of 10 possibilities for 320 total
solutions, if I did it right. The number of iterations is now only 5,040 and
you are evaluating fewer terms with c gone. The full test would have been
(10*a +b )+ (10*a + a) + (10*c + d) == (10*c + e)

BUT a tad more analysis shows that all solutions require "a" to be zero. If
a >=1 then 21*a must be >= 21.
But b and c cannot be more than 9+8 which is 17 and subtracting e makes it
no larger. So "a" MUST be zero.

So the solution can be done with only three loops for b, d, and e. 720
iterations.

The commented code is below. It can be done in as little as two lines of
code if the list comprehension is done in one line but why make it hard to
read.

If there is a flaw in my reasoning or the program below, please point it
out. You said there were 192 solutions. I found 320.
And, there are more efficient solutions possible but overkill for this
application.

Avi

################### START CODE ##############
# GOAL: find all solutions of a puzzle
# ab + aa + cd == ce
# Where each of the letters a through e
# are UNIQUE values ranging from 0 to 9

# Make the equation simpler
# expand ab to 10*a + b
# expand aa to 10*a + a
# expand cd to 10*c + d
# add to get 21*a + b + 10*c + d
# expand ce to 10*c + e
# simplify 21*a + b + 10*c + d = 10*c + e
# the 10*c on both sides cancel.
# RESULT: 21*a + b + d - e = 0
# the value of c is not relevant and
# you can solve without e and then add back
# all ten possibilities later.

# Method:
# Use a set of digits.
# Use a list comprehension with four loops.
# Each loop chooses all available values
# for a,b,d,e by subtracting the set
# of digits already in use at the time from all digits.

digits = set(range(10))

matches = [ (a, b, d, e)
            for a in digits
            for b in (digits - {a})
            for d in (digits -{a,b})
            for e in (digits -{a,b,d})
            if ( 21*a + b + d - e == 0)
            ]

print("SOLVING FOR: 21*a + b + d - e == 0")
print(f"matches found in batches of 10: {len(matches)}")

for (a,b,d,e) in matches:
    solution =  {'a' : a,
                    'b' : b,
                    'c' : 'any digit',
                    'd' : d,
                    'e' : e
                 }
    print(solution)

# Note the result shows all solutions have 'a' being zero.
# That was obvious from the equation as there were 21 of them
# and if a was 1 in 21*a + b + d - e = 0
# then clearly band d cannot be more than 9+8 so no solution
# unless a == 0.
# So you can solve this easier using the above techique by just
# solving b + d - e = 0

matches = [ (b, d, e)
            for b in digits
            for d in (digits -{b})
            for e in (digits -{b,d})
            if ( b + d == e)
            ]

print("\nSOLVING FOR: b + d == e")
print(f"matches found in batches of 10: {len(matches)}")

for (b,d,e) in matches:
    solution =  {'a' : '0',
                    'b' : b,
                    'c' : 'any digit',
                    'd' : d,
                    'e' : e
                 }
    print(solution)

################### END CODE ##############
################### BEGIN OUTPUT ##############
SOLVING FOR: 21*a + b + d - e == 0
matches found in batches of 10: 32
{'a': 0, 'b': 1, 'c': 'any digit', 'd': 2, 'e': 3}
{'a': 0, 'b': 1, 'c': 'any digit', 'd': 3, 'e': 4}
{'a': 0, 'b': 1, 'c': 'any digit', 'd': 4, 'e': 5}
{'a': 0, 'b': 1, 'c': 'any digit', 'd': 5, 'e': 6}
{'a': 0, 'b': 1, 'c': 'any digit', 'd': 6, 'e': 7}
{'a': 0, 'b': 1, 'c': 'any digit', 'd': 7, 'e': 8}
{'a': 0, 'b': 1, 'c': 'any digit', 'd': 8, 'e': 9}
{'a': 0, 'b': 2, 'c': 'any digit', 'd': 1, 'e': 3}
{'a': 0, 'b': 2, 'c': 'any digit', 'd': 3, 'e': 5}
{'a': 0, 'b': 2, 'c': 'any digit', 'd': 4, 'e': 6}
{'a': 0, 'b': 2, 'c': 'any digit', 'd': 5, 'e': 7}
{'a': 0, 'b': 2, 'c': 'any digit', 'd': 6, 'e': 8}
{'a': 0, 'b': 2, 'c': 'any digit', 'd': 7, 'e': 9}
{'a': 0, 'b': 3, 'c': 'any digit', 'd': 1, 'e': 4}
{'a': 0, 'b': 3, 'c': 'any digit', 'd': 2, 'e': 5}
{'a': 0, 'b': 3, 'c': 'any digit', 'd': 4, 'e': 7}
{'a': 0, 'b': 3, 'c': 'any digit', 'd': 5, 'e': 8}
{'a': 0, 'b': 3, 'c': 'any digit', 'd': 6, 'e': 9}
{'a': 0, 'b': 4, 'c': 'any digit', 'd': 1, 'e': 5}
{'a': 0, 'b': 4, 'c': 'any digit', 'd': 2, 'e': 6}
{'a': 0, 'b': 4, 'c': 'any digit', 'd': 3, 'e': 7}
{'a': 0, 'b': 4, 'c': 'any digit', 'd': 5, 'e': 9}
{'a': 0, 'b': 5, 'c': 'any digit', 'd': 1, 'e': 6}
{'a': 0, 'b': 5, 'c': 'any digit', 'd': 2, 'e': 7}
{'a': 0, 'b': 5, 'c': 'any digit', 'd': 3, 'e': 8}
{'a': 0, 'b': 5, 'c': 'any digit', 'd': 4, 'e': 9}
{'a': 0, 'b': 6, 'c': 'any digit', 'd': 1, 'e': 7}
{'a': 0, 'b': 6, 'c': 'any digit', 'd': 2, 'e': 8}
{'a': 0, 'b': 6, 'c': 'any digit', 'd': 3, 'e': 9}
{'a': 0, 'b': 7, 'c': 'any digit', 'd': 1, 'e': 8}
{'a': 0, 'b': 7, 'c': 'any digit', 'd': 2, 'e': 9}
{'a': 0, 'b': 8, 'c': 'any digit', 'd': 1, 'e': 9}

SOLVING FOR: b + d == e
matches found in batches of 10: 32
{'a': '0', 'b': 1, 'c': 'any digit', 'd': 2, 'e': 3}
{'a': '0', 'b': 1, 'c': 'any digit', 'd': 3, 'e': 4}
{'a': '0', 'b': 1, 'c': 'any digit', 'd': 4, 'e': 5}
{'a': '0', 'b': 1, 'c': 'any digit', 'd': 5, 'e': 6}
{'a': '0', 'b': 1, 'c': 'any digit', 'd': 6, 'e': 7}
{'a': '0', 'b': 1, 'c': 'any digit', 'd': 7, 'e': 8}
{'a': '0', 'b': 1, 'c': 'any digit', 'd': 8, 'e': 9}
{'a': '0', 'b': 2, 'c': 'any digit', 'd': 1, 'e': 3}
{'a': '0', 'b': 2, 'c': 'any digit', 'd': 3, 'e': 5}
{'a': '0', 'b': 2, 'c': 'any digit', 'd': 4, 'e': 6}
{'a': '0', 'b': 2, 'c': 'any digit', 'd': 5, 'e': 7}
{'a': '0', 'b': 2, 'c': 'any digit', 'd': 6, 'e': 8}
{'a': '0', 'b': 2, 'c': 'any digit', 'd': 7, 'e': 9}
{'a': '0', 'b': 3, 'c': 'any digit', 'd': 1, 'e': 4}
{'a': '0', 'b': 3, 'c': 'any digit', 'd': 2, 'e': 5}
{'a': '0', 'b': 3, 'c': 'any digit', 'd': 4, 'e': 7}
{'a': '0', 'b': 3, 'c': 'any digit', 'd': 5, 'e': 8}
{'a': '0', 'b': 3, 'c': 'any digit', 'd': 6, 'e': 9}
{'a': '0', 'b': 4, 'c': 'any digit', 'd': 1, 'e': 5}
{'a': '0', 'b': 4, 'c': 'any digit', 'd': 2, 'e': 6}
{'a': '0', 'b': 4, 'c': 'any digit', 'd': 3, 'e': 7}
{'a': '0', 'b': 4, 'c': 'any digit', 'd': 5, 'e': 9}
{'a': '0', 'b': 5, 'c': 'any digit', 'd': 1, 'e': 6}
{'a': '0', 'b': 5, 'c': 'any digit', 'd': 2, 'e': 7}
{'a': '0', 'b': 5, 'c': 'any digit', 'd': 3, 'e': 8}
{'a': '0', 'b': 5, 'c': 'any digit', 'd': 4, 'e': 9}
{'a': '0', 'b': 6, 'c': 'any digit', 'd': 1, 'e': 7}
{'a': '0', 'b': 6, 'c': 'any digit', 'd': 2, 'e': 8}
{'a': '0', 'b': 6, 'c': 'any digit', 'd': 3, 'e': 9}
{'a': '0', 'b': 7, 'c': 'any digit', 'd': 1, 'e': 8}
{'a': '0', 'b': 7, 'c': 'any digit', 'd': 2, 'e': 9}
{'a': '0', 'b': 8, 'c': 'any digit', 'd': 1, 'e': 9}
################### END OUTPUT ##############

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of jfong at ms4.hinet.net
Sent: Saturday, December 8, 2018 4:26 AM
To: python-list at python.org
Subject: Re: Why Python don't accept 03 as a number?

Avi Gross at 2018/12/8 UTC+8 PM2:09:20 wrote:
> [[READERS DIGEST CONDENSED ANSWER: use int("string") ]]
> 
> Since we all agree python will not make notations like "05" work 
> indefinitely, and the need expressed is how to solve a symbolic puzzle 
> (see message below) then it makes sense to look at alternate
representations.
> 
> I have a question first. How are you solving your puzzles? 
> 
>     ab + aa + cd == ce

Try all the combinations:-)

The only way I can think of is try-error. It takes no more 10 lines to go
from "ab + aa + cd == ce" to yield one correct answer, such as "03 + 00 + 15
== 18", using itertools' permutations(), string's translate() and re.

> Why does 05 ever even appear in your solution?

I don't know. There is total 192 answers for this puzzle anyway.

> Are you generating all possible answers by setting each variable to 
> one of 0 to 9 then the second to any of the remaining nine choices 
> then the third to the remaining 8 and so on? For any method like that, 
> you can presumably make each component like
> 
> ab = 10*a + b
> 
> in the loop.
> 
> Similarly for aa and cd and ce. If the equality above is true, you 
> found the solution and break out. If there can be multiple solutions, 
> note the solution and keep going. But note for the 5 variables above, 
> you are testing
> 10*9*8*7*6 combinations.
> 
> Another idea is to use strings like "05" as bizarrely, the function 
> int() seems to be an ex eption that does NOT care about leading 
> whitespace or
> zeroes:
> 
> >>> int("05")
> 5
> >>> int("  0005")
> 5
> 
> And even handles all zeroes:
> 
> >>> int("000000")
> 0
> 
> You can also use lstrip() with an argument to remove zeros:
> 
> >>> a = eval("05".lstrip("0"))
> 	     
> >>> a
> 	     
> 5
> 
> If you are in a situation where you only want to remove leading zeroes 
> if the following character is a digit and not "o" or "b" or "x", use 
> regular expressions or other techniques.

As far as the leading zero problem was concerned, the simplest way is using
re.sub()

> I will just toss in the possible use of the SymPy module to do actual 
> symbolic computations to solve some of these. Perhaps a tad advanced.

I don't know SymPy and if it can shorten the execution time. But I am very
curious about if there is other algorithm which can apply to this problem:-)

--Jach
--
https://mail.python.org/mailman/listinfo/python-list



From tomusatov at gmail.com  Sat Dec  8 13:02:41 2018
From: tomusatov at gmail.com (Musatov)
Date: Sat, 8 Dec 2018 10:02:41 -0800 (PST)
Subject: Program to keep track of success percentage
Message-ID: <6e487476-beb7-4968-b84d-9b2ce935b08c@googlegroups.com>

I am thinking about a program where the only user input is win/loss. The program let's you know if you have won more than 31% of the time or not. Any suggestions about how to approach authoring such a program? Thanks.


From alister.ware at ntlworld.com  Sat Dec  8 15:15:19 2018
From: alister.ware at ntlworld.com (Alister)
Date: Sat, 08 Dec 2018 20:15:19 GMT
Subject: Program to keep track of success percentage
References: <6e487476-beb7-4968-b84d-9b2ce935b08c@googlegroups.com>
Message-ID: <rDVOD.260253$xH4.78724@fx33.am4>

On Sat, 08 Dec 2018 10:02:41 -0800, Musatov wrote:

> I am thinking about a program where the only user input is win/loss. The
> program let's you know if you have won more than 31% of the time or not.
> Any suggestions about how to approach authoring such a program? Thanks.

To start describe how you would do it if keeping score on paper.

a simple loop
keep a count of the number for questions
keep a count of the number.
calculate the percentage.

once you have what is known as pseudo code you will be 90% of the way 
there
(python is effectively executable pseudo code anyway)

this should be more than enough to help you with you homework without 
doing it for you




-- 
If reporters don't know that truth is plural, they ought to be lawyers.
		-- Tom Wicker


From ian.g.kelly at gmail.com  Sat Dec  8 15:41:09 2018
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 8 Dec 2018 13:41:09 -0700
Subject: Program to keep track of success percentage
In-Reply-To: <rDVOD.260253$xH4.78724@fx33.am4>
References: <6e487476-beb7-4968-b84d-9b2ce935b08c@googlegroups.com>
 <rDVOD.260253$xH4.78724@fx33.am4>
Message-ID: <CALwzidm-QdXR55OqVa_Oa2=50z13A7Pn3t1HpY2defrhhT_NiQ@mail.gmail.com>

On Sat, Dec 8, 2018 at 1:22 PM Alister via Python-list
<python-list at python.org> wrote:
>
> On Sat, 08 Dec 2018 10:02:41 -0800, Musatov wrote:
>
> > I am thinking about a program where the only user input is win/loss. The
> > program let's you know if you have won more than 31% of the time or not.
> > Any suggestions about how to approach authoring such a program? Thanks.
>
> To start describe how you would do it if keeping score on paper.
>
> a simple loop
> keep a count of the number for questions
> keep a count of the number.
> calculate the percentage.
>
> once you have what is known as pseudo code you will be 90% of the way
> there
> (python is effectively executable pseudo code anyway)
>
> this should be more than enough to help you with you homework without
> doing it for you

If this is not homework on the other hand, then my recommendation is
to just use a spreadsheet.


From python.list at tim.thechases.com  Sat Dec  8 15:59:38 2018
From: python.list at tim.thechases.com (Tim Chase)
Date: Sat, 8 Dec 2018 14:59:38 -0600
Subject: Program to keep track of success percentage
In-Reply-To: <6e487476-beb7-4968-b84d-9b2ce935b08c@googlegroups.com>
References: <6e487476-beb7-4968-b84d-9b2ce935b08c@googlegroups.com>
Message-ID: <20181208145938.6d20bc8f@bigbox.christie.dr>

On 2018-12-08 10:02, Musatov wrote:
> I am thinking about a program where the only user input is
> win/loss. The program let's you know if you have won more than 31%
> of the time or not. Any suggestions about how to approach authoring
> such a program? Thanks. --

Can be done with an awk one-liner:

awk '/[wW]/{w+=1}{printf("W: %i L: %i %i%%\n", w, NR-w, w *
100/NR)}END{if (w * 100/NR > 31) print "More than 31% winning"}'

-tkc





From HooDunnit at didly42KahZidly.net  Sat Dec  8 16:01:32 2018
From: HooDunnit at didly42KahZidly.net (Cousin Stanley)
Date: Sat, 08 Dec 2018 14:01:32 -0700
Subject: Program to keep track of success percentage
References: <6e487476-beb7-4968-b84d-9b2ce935b08c@googlegroups.com>
Message-ID: <puhbff$98h$1@dont-email.me>

Musatov wrote:

> I am thinking about a program where the only user input is win/loss. 
> 
> The program let's you know if you have won 
> more than 31% of the time or not. 
> 
> Any suggestions about how to approach authoring
> such a program? Thanks.

  The following results are from a python  toss_up  program
  using the python random.choice module where  win  or  lose  
  was chosen randomly for each try for various numbers of tries ....

  For random choice of  win  or  lose 
  distribution seems to be very even 
  for each result .... 

     # tries       win %       lose %

        10         50.00        50.00
       100         52.00        48.00
      1000         48.40        51.60
     10000         49.75        50.25
    100000         50.17        49.83
   1000000         50.00        50.00


-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona



From Richard at Damon-Family.org  Sat Dec  8 17:29:56 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Sat, 8 Dec 2018 17:29:56 -0500
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <mailman.1.1544290812.2771.python-list@python.org>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
 <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <002401d48ebc$82f70690$88e513b0$@verizon.net>
 <mailman.96.1544249346.25456.python-list@python.org>
 <f79c5efa-5baf-41ed-92bc-89257a8ecd87@googlegroups.com>
 <005701d48f1d$0f253ff0$2d6fbfd0$@verizon.net>
 <mailman.1.1544290812.2771.python-list@python.org>
Message-ID: <EBXOD.729543$8r2.86205@fx34.iad>

On 12/8/18 12:40 PM, Avi Gross wrote:
> You are solving for: ab + aa + cd == ce

Actually, an even quicker analysis for this particular problem is:

from the 10s digits, a + a + c + carryin = c
Thus a and carryin must both be 0 (carryin can not be negative, nor any
of the variables)

thus the final solution space is:

b + d = e
a = 0
c any other digit (6 possible values for every combo of b, d, e)

if b is <= 4, there are 8-b possible values of d that will have a legal
value of e.
b = 1, we get d = 2, 3, 4, ... 7, 8
b = 2, we get d = 1, 3, 4, ... 6, 7 (8 would generate carry)
b = 3, we get d = 1, 2, 4, 5, 6
b = 4, we get d = 1, 2, 3, 5

if b >= 5 we get 9-b possible values of d (we no longer have to omit the
possible value of b  = d)

So the number of possible answers are:

(7+6+5+4+4+3+2+1)*6 = 192

(your 320 was you gave c 10 possible values, but you need to remove the
duplicates).


From avigross at verizon.net  Sat Dec  8 17:54:07 2018
From: avigross at verizon.net (Avi Gross)
Date: Sat, 8 Dec 2018 17:54:07 -0500
Subject: Program to keep track of success percentage
In-Reply-To: <20181208145938.6d20bc8f@bigbox.christie.dr>
References: <6e487476-beb7-4968-b84d-9b2ce935b08c@googlegroups.com>
 <20181208145938.6d20bc8f@bigbox.christie.dr>
Message-ID: <000601d48f48$ed1969f0$c74c3dd0$@verizon.net>

Tim,

This may be a bit awkward.

I am not sure a question on the python list expects to get a one-liner, let alone in an unrelated language like AWK. Unless you install other environments like Cygwin, AWK does not tend to be available of platforms like Windows. Ditto for PERL and other languages or tools you want to use this way.

There are times people appreciate something sophisticated or clever or even innovative.

My personal take on this question is of someone who does not necessarily know what they need to do, let alone in python. And they did not specify how the data is supplied or whether it is a one-time thing.

The two numbers needed could be supplied on the command line and gotten using argv or gotten using two input statements or read from a file or stdin. Or they could be asking for a cumulative set of wins/losses to be contiuously evaluated and as soon as the threshold of 31% is attained, print a message. OR, there could be a previously agreed upon number of games, such as 164, and you get win/loss criteria some way and you want the percentage of wins divided by the 164 to be 31%. 

My GUESS is they are getting a stream of win/win/loss/win/loss/... type of data, maybe one per line of text, maybe as 0 versus 1, but who knows. If we knew exactly, it might lead to trivial solutions. For example, if they had a series of 0/1 you could do a sum() and a count() and divide. 

But what if the scenario was to provide a score like 5 - 3 using whatever notation. Now you need to figure out if your side won and perhaps deal with ties before counting and dividing.

Too many scenarios are possible but the simple answer is more like someone else said.

Count the number of wins, or keep track of it.
Count the total you need.
Divide one by the other and compare it to 0.31 or 31 depending on your calculation method.

Your solution in AWK assumes  lots of things. You assume the data is either on stdin or comes from automatically opening file names on the command line to generate a stream of lines. You assume a win is any line containing one or more lower or upper case instances of the letter W. You let AWK count lines and store that in NR and assume anything without a W is a loss. You print a running commentary and  only at the end of input state if they exceeded 31%.

Now that is quite possibly a way to go. But the data may not be set up that way or they may want to quit as soon as the threshold is reached or there may be blank lines or a notation for a tie. Some of those may not be one-liners. Yours would read much better if spaced out, but you might have written it this way when you were ?

But, as mentioned, this is a python board. Would you care to do the same thing as a brief program in that language.

If we had that pesky := operator, maybe. 

Just kidding.

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Tim Chase
Sent: Saturday, December 8, 2018 4:00 PM
To: Musatov <tomusatov at gmail.com>
Cc: python-list at python.org
Subject: Re: Program to keep track of success percentage

On 2018-12-08 10:02, Musatov wrote:
> I am thinking about a program where the only user input is win/loss. 
> The program let's you know if you have won more than 31% of the time 
> or not. Any suggestions about how to approach authoring such a 
> program? Thanks. --

Can be done with an awk one-liner:

awk '/[wW]/{w+=1}{printf("W: %i L: %i %i%%\n", w, NR-w, w * 100/NR)}END{if (w * 100/NR > 31) print "More than 31% winning"}'

-tkc



--
https://mail.python.org/mailman/listinfo/python-list



From avigross at verizon.net  Sat Dec  8 18:23:36 2018
From: avigross at verizon.net (Avi Gross)
Date: Sat, 8 Dec 2018 18:23:36 -0500
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <EBXOD.729543$8r2.86205@fx34.iad>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
 <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <002401d48ebc$82f70690$88e513b0$@verizon.net>
 <mailman.96.1544249346.25456.python-list@python.org>
 <f79c5efa-5baf-41ed-92bc-89257a8ecd87@googlegroups.com>
 <005701d48f1d$0f253ff0$2d6fbfd0$@verizon.net>
 <mailman.1.1544290812.2771.python-list@python.org>
 <EBXOD.729543$8r2.86205@fx34.iad>
Message-ID: <000a01d48f4d$0ba12170$22e36450$@verizon.net>

[DISCLAIMER: less about python than analysis of a puzzle]

Richard,

Thank you for pointing out that c in the puzzle is constrained. That
explains why my 320 answers are too many. It cannot be 0 as "a" is always
zero and it cannot be the three other values that b,d,e are using at the
time. So my earlier solution should say c is any of six available choices. 

So 6 * 32 is 192 solutions.

Your analysis requires considering a carry from the right column into the
left. I skipped that by converting something like ab to 10*a+b.

This all began with someone trying to enter 03 which suggests they may have
been starting to look at it your way. As a logic puzzle to do those with
pencil and paper, your analysis is spot on. There is additional info to be
gleaned by looking at adding columns. I chose a perspective on more brute
force methods and whittled it down to less forceful. Given that c was
removed from the equation, though, and that only 6 of 10 options are
available for any given time, I would need another pass at the answers and
for each solution for the others (b,d,e if we agree a is always 0) I would
need to make six entries with c set successively to digits-set(0,b,d,e} or
something like that.

I note that programming some of the kinds of analysis some of these puzzles
use is not as easy in programming languages as a more brute-force approach
that computers are better at than humans. 

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Richard Damon
Sent: Saturday, December 8, 2018 5:30 PM
To: python-list at python.org
Subject: Re: Why Python don't accept 03 as a number?

On 12/8/18 12:40 PM, Avi Gross wrote:
> You are solving for: ab + aa + cd == ce

Actually, an even quicker analysis for this particular problem is:

from the 10s digits, a + a + c + carryin = c Thus a and carryin must both be
0 (carryin can not be negative, nor any of the variables)

thus the final solution space is:

b + d = e
a = 0
c any other digit (6 possible values for every combo of b, d, e)

if b is <= 4, there are 8-b possible values of d that will have a legal
value of e.
b = 1, we get d = 2, 3, 4, ... 7, 8
b = 2, we get d = 1, 3, 4, ... 6, 7 (8 would generate carry) b = 3, we get d
= 1, 2, 4, 5, 6 b = 4, we get d = 1, 2, 3, 5

if b >= 5 we get 9-b possible values of d (we no longer have to omit the
possible value of b  = d)

So the number of possible answers are:

(7+6+5+4+4+3+2+1)*6 = 192

(your 320 was you gave c 10 possible values, but you need to remove the
duplicates).
--
https://mail.python.org/mailman/listinfo/python-list



From python.list at tim.thechases.com  Sat Dec  8 19:09:53 2018
From: python.list at tim.thechases.com (Tim Chase)
Date: Sat, 8 Dec 2018 18:09:53 -0600
Subject: Program to keep track of success percentage
In-Reply-To: <000601d48f48$ed1969f0$c74c3dd0$@verizon.net>
References: <6e487476-beb7-4968-b84d-9b2ce935b08c@googlegroups.com>
 <20181208145938.6d20bc8f@bigbox.christie.dr>
 <000601d48f48$ed1969f0$c74c3dd0$@verizon.net>
Message-ID: <20181208180953.2af3acba@bigbox.christie.dr>

On 2018-12-08 17:54, Avi Gross wrote:
> This may be a bit awkward.

ICWYDT. "awk"ward. :wide-eyed_gaping_grin_with_finger-guns:

You seem to have your knickers in a knot.

> Your solution in AWK assumes  lots of things. You assume the data
> is either on stdin or comes from automatically opening file names
> on the command line to generate a stream of lines. You assume a win
> is any line containing one or more lower or upper case instances of
> the letter W. You let AWK count lines and store that in NR and
> assume anything without a W is a loss. You print a running
> commentary and  only at the end of input state if they exceeded 31%.

1) yes the problem was underdefined.  If all they want is to is tally
wins ("the only user input is win/loss". Note: not blank. Not ties.
Not garbage. Not "victorias/derrotas" nor "?????/????????"), the awk
one-liner does exactly that.  I'll grant, that the OP didn't specify
whether this was on Windows, Linux, Mac, BSD, DOS, ULTRIX, or
anything at all about the operating system.  The concepts for the
solution remain, even if awk is unavailable.  If the input isn't from
stdin it's not standard and should be specified in the problem-space
(there's a reason it's called *standard input*).

2) the problem sounded an awful lot like homework.  I'm not going to
answer a *Python* homework problem in *Python*.  I'm willing to give
the solution in another language (done) so the OP can translate
those ideas into Python.  I'm also willing to take what the OP has
already written (nothing provided in the original email) and help the
OP iterate with that.  The original request, while posted to the
Python mailing list, didn't even specify that it had to be in
Python.  If it's not homework, then the one-liner solves their
problem on any modern platform with a CLI that isn't Windows (and
even on Windows if one installs a version of awk there.)

Yes.  It could also have had a sqlite/mysql/postgres database
back-end, and command-line interface for tracking wins/losses, and a
web front-end for displaying statistics and reporting wins/losses,
and a tkinter/wx/whatever GUI for team management, and export PDFs,
and use TensorFlow for AI analysis of the results.  But that's not
what the OP asked for.

> Yours would read much better if spaced out, but you might have
> written it this way when you were ?

While I was not in any chemically-altered state of mind, while I
penned it as one line, it would certainly be more readable as a
script.

#!env awk -f
/[wW]/{
 w += 1;
}
{
 printf("W: %i L: %i %i%%\n", w, NR-w, w * 100/NR);
}
END {
 if (w * 100/NR > 31)
  print "More than 31% winning"
}


There.  Happy?

> Would you care to do the same thing as a brief program in that
> language.

I can (and for the record, did), but I don't provide other people
with the prefab answers to their homework.

-tkc










From Richard at Damon-Family.org  Sat Dec  8 19:34:33 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Sat, 8 Dec 2018 19:34:33 -0500
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <000a01d48f4d$0ba12170$22e36450$@verizon.net>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
 <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <002401d48ebc$82f70690$88e513b0$@verizon.net>
 <mailman.96.1544249346.25456.python-list@python.org>
 <f79c5efa-5baf-41ed-92bc-89257a8ecd87@googlegroups.com>
 <005701d48f1d$0f253ff0$2d6fbfd0$@verizon.net>
 <mailman.1.1544290812.2771.python-list@python.org>
 <EBXOD.729543$8r2.86205@fx34.iad>
 <000a01d48f4d$0ba12170$22e36450$@verizon.net>
Message-ID: <d919439e-3be3-ca42-d2ce-2d4a339f8eab@Damon-Family.org>

On 12/8/18 6:23 PM, Avi Gross wrote:
> [DISCLAIMER: less about python than analysis of a puzzle]
>
> Richard,
>
> Thank you for pointing out that c in the puzzle is constrained. That
> explains why my 320 answers are too many. It cannot be 0 as "a" is always
> zero and it cannot be the three other values that b,d,e are using at the
> time. So my earlier solution should say c is any of six available choices. 
>
> So 6 * 32 is 192 solutions.
>
> Your analysis requires considering a carry from the right column into the
> left. I skipped that by converting something like ab to 10*a+b.
>
> This all began with someone trying to enter 03 which suggests they may have
> been starting to look at it your way. As a logic puzzle to do those with
> pencil and paper, your analysis is spot on. There is additional info to be
> gleaned by looking at adding columns. I chose a perspective on more brute
> force methods and whittled it down to less forceful. Given that c was
> removed from the equation, though, and that only 6 of 10 options are
> available for any given time, I would need another pass at the answers and
> for each solution for the others (b,d,e if we agree a is always 0) I would
> need to make six entries with c set successively to digits-set(0,b,d,e} or
> something like that.
>
> I note that programming some of the kinds of analysis some of these puzzles
> use is not as easy in programming languages as a more brute-force approach
> that computers are better at than humans. 
Actually, part of my point was that we are often tempted to go simple
brute force when a bit of analysis can often vastly simplify the problem
to be solved.

Now, the brute force program, if done right, could also handle other
related problems like SEND+MORE=MONEY

But now, the input equation isn't controlled by the program, so you need
to be very careful about using it for something that gets sent to eval,
and by the time you take that care, perhaps it is easier to do the safer
way.

-- 
Richard Damon



From jfong at ms4.hinet.net  Sat Dec  8 21:51:00 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Sat, 8 Dec 2018 18:51:00 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <mailman.0.1544287910.2771.python-list@python.org>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net> <pugsmp$i6a$1@blaine.gmane.org>
 <mailman.0.1544287910.2771.python-list@python.org>
Message-ID: <f323fc6f-4d3d-4dcb-a0b5-bdff3d5e8617@googlegroups.com>

Grant Edwards? 2018?12?9???? UTC+8??12?52?04????
> On 2018-12-08, Cameron Simpson <cs at cskk.id.au> wrote:
> > On 07Dec2018 20:24, Jach Fong <jfong at ms4.hinet.net> wrote:
> >>Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
> >>> What is it exactly that you're trying to accomplish with this? Perhaps
> >>> there's a better way than using eval.
> >>
> >>This problem comes from solving a word puzzle,
> >>    ab + aa + cd == ce
> >>Each character will be translate to a digit and evaluate the correctness,
> >>    03 + 00 + 15 == 18
> >
> > Then you should be evaluating the digits and assembling values from 
> > them. Not trying to shoehorn a string through something that _might_ 
> > accept this string and do what you want. In Python 2 it will accept your 
> > string and not do what you want; at least in Python 3 it doesn't accept 
> > your string.
> 
> Just to be clear: you do _not_ want to use eval on the string.
> 
> If you're not the one who created the string, it might wipe your hard
> drive or empty your bank account.  If you _are_ the one who created
> the string, then generate the desired result instead.
> 
> -- 
> Grant

I didn't evaluate the input string directly. It's the translated "digit" string been evaluated, so shouldn't have any danger on using eval().

--Jach


From jfong at ms4.hinet.net  Sat Dec  8 21:55:56 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Sat, 8 Dec 2018 18:55:56 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <mailman.1.1544290812.2771.python-list@python.org>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <e4527da9-816b-7d77-c4c7-f4d68926068a@mrabarnett.plus.com>
 <mailman.91.1544234677.25456.python-list@python.org>
 <2a04e7fc-0db9-4858-a068-fde706dd5378@googlegroups.com>
 <CALwzidkVj+=6cowmaH9PSGZdK=RdqJDOg0QPbZnxeCVdS5+9UA@mail.gmail.com>
 <mailman.93.1544239701.25456.python-list@python.org>
 <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <002401d48ebc$82f70690$88e513b0$@verizon.net>
 <mailman.96.1544249346.25456.python-list@python.org>
 <f79c5efa-5baf-41ed-92bc-89257a8ecd87@googlegroups.com>
 <005701d48f1d$0f253ff0$2d6fbfd0$@verizon.net>
 <mailman.1.1544290812.2771.python-list@python.org>
Message-ID: <6f6eb330-47a4-4acc-86bb-12964f072ded@googlegroups.com>

Avi Gross? 2018?12?9???? UTC+8??1?40?26????
> Jach,
> 
> Just for fun, I looked at the  puzzle you asked about and solved it several
> ways without running into your 03 problem at all. There are more efficient
> solutions than total brute force.
> 
> Anyone not interested, stop here, please. After my explanations, I show a
> python program then the output it produces.
> 
> The comments in the python program (below) explain the solution classes but
> let me suggest a little algebra simplifies the problem so fewer brute force
> iterations are needed.
> 
> The simplistic algorithm can be written as a single list comprehension with
> (in this case) 5 "for" clauses and one "if" clause.
> 
> You are solving for: ab + aa + cd == ce
> 
> So 
> for all digits possible for a
> for all remaining digits possible for b each iteration
> for all remaining digits possible for c each iteration
> for all remaining digits possible for d each iteration
> for all remaining digits possible for e each iteration
> if the condition applies.
> 
> Clearly the above is deeply nested with 10!/5! Iterations or 30,240.
> 
> But a little algebra simplifies the solution so c drops out of the equation
> as shown below in the comments.
> The test becomes:  21*a + b + d - e == 0
> 
> You can apply that as the condition using four loops. You get 32 solutions
> and for each you can let c be any of 10 possibilities for 320 total
> solutions, if I did it right. The number of iterations is now only 5,040 and
> you are evaluating fewer terms with c gone. The full test would have been
> (10*a +b )+ (10*a + a) + (10*c + d) == (10*c + e)
> 
> BUT a tad more analysis shows that all solutions require "a" to be zero. If
> a >=1 then 21*a must be >= 21.
> But b and c cannot be more than 9+8 which is 17 and subtracting e makes it
> no larger. So "a" MUST be zero.
> 
> So the solution can be done with only three loops for b, d, and e. 720
> iterations.
> 
> The commented code is below. It can be done in as little as two lines of
> code if the list comprehension is done in one line but why make it hard to
> read.
> 
> If there is a flaw in my reasoning or the program below, please point it
> out. You said there were 192 solutions. I found 320.
> And, there are more efficient solutions possible but overkill for this
> application.
> 
> Avi
> 
> ################### START CODE ##############
> # GOAL: find all solutions of a puzzle
> # ab + aa + cd == ce
> # Where each of the letters a through e
> # are UNIQUE values ranging from 0 to 9
> 
> # Make the equation simpler
> # expand ab to 10*a + b
> # expand aa to 10*a + a
> # expand cd to 10*c + d
> # add to get 21*a + b + 10*c + d
> # expand ce to 10*c + e
> # simplify 21*a + b + 10*c + d = 10*c + e
> # the 10*c on both sides cancel.
> # RESULT: 21*a + b + d - e = 0
> # the value of c is not relevant and
> # you can solve without e and then add back
> # all ten possibilities later.
> 
> # Method:
> # Use a set of digits.
> # Use a list comprehension with four loops.
> # Each loop chooses all available values
> # for a,b,d,e by subtracting the set
> # of digits already in use at the time from all digits.
> 
> digits = set(range(10))
> 
> matches = [ (a, b, d, e)
>             for a in digits
>             for b in (digits - {a})
>             for d in (digits -{a,b})
>             for e in (digits -{a,b,d})
>             if ( 21*a + b + d - e == 0)
>             ]
> 
> print("SOLVING FOR: 21*a + b + d - e == 0")
> print(f"matches found in batches of 10: {len(matches)}")
> 
> for (a,b,d,e) in matches:
>     solution =  {'a' : a,
>                     'b' : b,
>                     'c' : 'any digit',
>                     'd' : d,
>                     'e' : e
>                  }
>     print(solution)
> 
> # Note the result shows all solutions have 'a' being zero.
> # That was obvious from the equation as there were 21 of them
> # and if a was 1 in 21*a + b + d - e = 0
> # then clearly band d cannot be more than 9+8 so no solution
> # unless a == 0.
> # So you can solve this easier using the above techique by just
> # solving b + d - e = 0
> 
> matches = [ (b, d, e)
>             for b in digits
>             for d in (digits -{b})
>             for e in (digits -{b,d})
>             if ( b + d == e)
>             ]
> 
> print("\nSOLVING FOR: b + d == e")
> print(f"matches found in batches of 10: {len(matches)}")
> 
> for (b,d,e) in matches:
>     solution =  {'a' : '0',
>                     'b' : b,
>                     'c' : 'any digit',
>                     'd' : d,
>                     'e' : e
>                  }
>     print(solution)
> 
> ################### END CODE ##############
> ################### BEGIN OUTPUT ##############
> SOLVING FOR: 21*a + b + d - e == 0
> matches found in batches of 10: 32
> {'a': 0, 'b': 1, 'c': 'any digit', 'd': 2, 'e': 3}
> {'a': 0, 'b': 1, 'c': 'any digit', 'd': 3, 'e': 4}
> {'a': 0, 'b': 1, 'c': 'any digit', 'd': 4, 'e': 5}
> {'a': 0, 'b': 1, 'c': 'any digit', 'd': 5, 'e': 6}
> {'a': 0, 'b': 1, 'c': 'any digit', 'd': 6, 'e': 7}
> {'a': 0, 'b': 1, 'c': 'any digit', 'd': 7, 'e': 8}
> {'a': 0, 'b': 1, 'c': 'any digit', 'd': 8, 'e': 9}
> {'a': 0, 'b': 2, 'c': 'any digit', 'd': 1, 'e': 3}
> {'a': 0, 'b': 2, 'c': 'any digit', 'd': 3, 'e': 5}
> {'a': 0, 'b': 2, 'c': 'any digit', 'd': 4, 'e': 6}
> {'a': 0, 'b': 2, 'c': 'any digit', 'd': 5, 'e': 7}
> {'a': 0, 'b': 2, 'c': 'any digit', 'd': 6, 'e': 8}
> {'a': 0, 'b': 2, 'c': 'any digit', 'd': 7, 'e': 9}
> {'a': 0, 'b': 3, 'c': 'any digit', 'd': 1, 'e': 4}
> {'a': 0, 'b': 3, 'c': 'any digit', 'd': 2, 'e': 5}
> {'a': 0, 'b': 3, 'c': 'any digit', 'd': 4, 'e': 7}
> {'a': 0, 'b': 3, 'c': 'any digit', 'd': 5, 'e': 8}
> {'a': 0, 'b': 3, 'c': 'any digit', 'd': 6, 'e': 9}
> {'a': 0, 'b': 4, 'c': 'any digit', 'd': 1, 'e': 5}
> {'a': 0, 'b': 4, 'c': 'any digit', 'd': 2, 'e': 6}
> {'a': 0, 'b': 4, 'c': 'any digit', 'd': 3, 'e': 7}
> {'a': 0, 'b': 4, 'c': 'any digit', 'd': 5, 'e': 9}
> {'a': 0, 'b': 5, 'c': 'any digit', 'd': 1, 'e': 6}
> {'a': 0, 'b': 5, 'c': 'any digit', 'd': 2, 'e': 7}
> {'a': 0, 'b': 5, 'c': 'any digit', 'd': 3, 'e': 8}
> {'a': 0, 'b': 5, 'c': 'any digit', 'd': 4, 'e': 9}
> {'a': 0, 'b': 6, 'c': 'any digit', 'd': 1, 'e': 7}
> {'a': 0, 'b': 6, 'c': 'any digit', 'd': 2, 'e': 8}
> {'a': 0, 'b': 6, 'c': 'any digit', 'd': 3, 'e': 9}
> {'a': 0, 'b': 7, 'c': 'any digit', 'd': 1, 'e': 8}
> {'a': 0, 'b': 7, 'c': 'any digit', 'd': 2, 'e': 9}
> {'a': 0, 'b': 8, 'c': 'any digit', 'd': 1, 'e': 9}
> 
> SOLVING FOR: b + d == e
> matches found in batches of 10: 32
> {'a': '0', 'b': 1, 'c': 'any digit', 'd': 2, 'e': 3}
> {'a': '0', 'b': 1, 'c': 'any digit', 'd': 3, 'e': 4}
> {'a': '0', 'b': 1, 'c': 'any digit', 'd': 4, 'e': 5}
> {'a': '0', 'b': 1, 'c': 'any digit', 'd': 5, 'e': 6}
> {'a': '0', 'b': 1, 'c': 'any digit', 'd': 6, 'e': 7}
> {'a': '0', 'b': 1, 'c': 'any digit', 'd': 7, 'e': 8}
> {'a': '0', 'b': 1, 'c': 'any digit', 'd': 8, 'e': 9}
> {'a': '0', 'b': 2, 'c': 'any digit', 'd': 1, 'e': 3}
> {'a': '0', 'b': 2, 'c': 'any digit', 'd': 3, 'e': 5}
> {'a': '0', 'b': 2, 'c': 'any digit', 'd': 4, 'e': 6}
> {'a': '0', 'b': 2, 'c': 'any digit', 'd': 5, 'e': 7}
> {'a': '0', 'b': 2, 'c': 'any digit', 'd': 6, 'e': 8}
> {'a': '0', 'b': 2, 'c': 'any digit', 'd': 7, 'e': 9}
> {'a': '0', 'b': 3, 'c': 'any digit', 'd': 1, 'e': 4}
> {'a': '0', 'b': 3, 'c': 'any digit', 'd': 2, 'e': 5}
> {'a': '0', 'b': 3, 'c': 'any digit', 'd': 4, 'e': 7}
> {'a': '0', 'b': 3, 'c': 'any digit', 'd': 5, 'e': 8}
> {'a': '0', 'b': 3, 'c': 'any digit', 'd': 6, 'e': 9}
> {'a': '0', 'b': 4, 'c': 'any digit', 'd': 1, 'e': 5}
> {'a': '0', 'b': 4, 'c': 'any digit', 'd': 2, 'e': 6}
> {'a': '0', 'b': 4, 'c': 'any digit', 'd': 3, 'e': 7}
> {'a': '0', 'b': 4, 'c': 'any digit', 'd': 5, 'e': 9}
> {'a': '0', 'b': 5, 'c': 'any digit', 'd': 1, 'e': 6}
> {'a': '0', 'b': 5, 'c': 'any digit', 'd': 2, 'e': 7}
> {'a': '0', 'b': 5, 'c': 'any digit', 'd': 3, 'e': 8}
> {'a': '0', 'b': 5, 'c': 'any digit', 'd': 4, 'e': 9}
> {'a': '0', 'b': 6, 'c': 'any digit', 'd': 1, 'e': 7}
> {'a': '0', 'b': 6, 'c': 'any digit', 'd': 2, 'e': 8}
> {'a': '0', 'b': 6, 'c': 'any digit', 'd': 3, 'e': 9}
> {'a': '0', 'b': 7, 'c': 'any digit', 'd': 1, 'e': 8}
> {'a': '0', 'b': 7, 'c': 'any digit', 'd': 2, 'e': 9}
> {'a': '0', 'b': 8, 'c': 'any digit', 'd': 1, 'e': 9}
> ################### END OUTPUT ##############
> 
> -----Original Message-----
> From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
> Behalf Of jfong at ms4.hinet.net
> Sent: Saturday, December 8, 2018 4:26 AM
> To: python-list at python.org
> Subject: Re: Why Python don't accept 03 as a number?
> 
> Avi Gross at 2018/12/8 UTC+8 PM2:09:20 wrote:
> > [[READERS DIGEST CONDENSED ANSWER: use int("string") ]]
> > 
> > Since we all agree python will not make notations like "05" work 
> > indefinitely, and the need expressed is how to solve a symbolic puzzle 
> > (see message below) then it makes sense to look at alternate
> representations.
> > 
> > I have a question first. How are you solving your puzzles? 
> > 
> >     ab + aa + cd == ce
> 
> Try all the combinations:-)
> 
> The only way I can think of is try-error. It takes no more 10 lines to go
> from "ab + aa + cd == ce" to yield one correct answer, such as "03 + 00 + 15
> == 18", using itertools' permutations(), string's translate() and re.
> 
> > Why does 05 ever even appear in your solution?
> 
> I don't know. There is total 192 answers for this puzzle anyway.
> 
> > Are you generating all possible answers by setting each variable to 
> > one of 0 to 9 then the second to any of the remaining nine choices 
> > then the third to the remaining 8 and so on? For any method like that, 
> > you can presumably make each component like
> > 
> > ab = 10*a + b
> > 
> > in the loop.
> > 
> > Similarly for aa and cd and ce. If the equality above is true, you 
> > found the solution and break out. If there can be multiple solutions, 
> > note the solution and keep going. But note for the 5 variables above, 
> > you are testing
> > 10*9*8*7*6 combinations.
> > 
> > Another idea is to use strings like "05" as bizarrely, the function 
> > int() seems to be an ex eption that does NOT care about leading 
> > whitespace or
> > zeroes:
> > 
> > >>> int("05")
> > 5
> > >>> int("  0005")
> > 5
> > 
> > And even handles all zeroes:
> > 
> > >>> int("000000")
> > 0
> > 
> > You can also use lstrip() with an argument to remove zeros:
> > 
> > >>> a = eval("05".lstrip("0"))
> > 	     
> > >>> a
> > 	     
> > 5
> > 
> > If you are in a situation where you only want to remove leading zeroes 
> > if the following character is a digit and not "o" or "b" or "x", use 
> > regular expressions or other techniques.
> 
> As far as the leading zero problem was concerned, the simplest way is using
> re.sub()
> 
> > I will just toss in the possible use of the SymPy module to do actual 
> > symbolic computations to solve some of these. Perhaps a tad advanced.
> 
> I don't know SymPy and if it can shorten the execution time. But I am very
> curious about if there is other algorithm which can apply to this problem:-)
> 
> --Jach
> --
> https://mail.python.org/mailman/listinfo/python-list

I appreciate your analysis of this particular equation. Had learn something from it, especially the way the "matches" does. Thank you.

The actual equation is from the user input (sorry didn't mention it early) so can't do any analysis ahead, and also don't know how to write codes to do it too. The only way left to me is the "brute force" :-(

--Jach


From memilanuk at gmail.com  Sat Dec  8 22:12:46 2018
From: memilanuk at gmail.com (Monte Milanuk)
Date: Sat, 8 Dec 2018 19:12:46 -0800
Subject: conda/anaconda and pip3 (pip)
In-Reply-To: <pu3po6$2t8$1@gioia.aioe.org>
References: <pu3po6$2t8$1@gioia.aioe.org>
Message-ID: <pui13b$q1i$1@blaine.gmane.org>

Did you find any solution(s)?



From ian.g.kelly at gmail.com  Sat Dec  8 22:57:10 2018
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Sat, 8 Dec 2018 20:57:10 -0700
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <f323fc6f-4d3d-4dcb-a0b5-bdff3d5e8617@googlegroups.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net> <pugsmp$i6a$1@blaine.gmane.org>
 <mailman.0.1544287910.2771.python-list@python.org>
 <f323fc6f-4d3d-4dcb-a0b5-bdff3d5e8617@googlegroups.com>
Message-ID: <CALwzidnG3eHL0m+6dD2W48dyNnH-tOws1s2Ws+1mr5Mp2zTMQg@mail.gmail.com>

On Sat, Dec 8, 2018 at 7:57 PM <jfong at ms4.hinet.net> wrote:
>
> Grant Edwards? 2018?12?9???? UTC+8??12?52?04????
> > Just to be clear: you do _not_ want to use eval on the string.
> >
> > If you're not the one who created the string, it might wipe your hard
> > drive or empty your bank account.  If you _are_ the one who created
> > the string, then generate the desired result instead.
> >
> > --
> > Grant
>
> I didn't evaluate the input string directly. It's the translated "digit" string been evaluated, so shouldn't have any danger on using eval().

Replacing the first five letters of the alphabet is not sufficient to
sanitize untrusted input for eval. Here's a simple example that avoids
using any of those letters:

py> eval(re.sub(r'[a-e]', '0',
"__import__('su\\x62pro\\x63\\x65ss').run('\\x65\\x63ho rm -rf /',
**{'sh\\x65ll': 1})"))
rm -rf /
CompletedProcess(args='echo rm -rf /', returncode=0)

Now, if you remove *all* the characters that could possibly start
identifiers (https://docs.python.org/3/reference/lexical_analysis.html#identifiers)
then you might be safe. Possibly just removing all the ones in ASCII
(A-Z + a-z + _) would suffice. I make no guarantees either way.

I wish I could say you should just use ast.literal_eval instead.
Unfortunately it doesn't seem to support ==:

py> ast.literal_eval('10 + 20 == 30')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/ast.py", line 84, in literal_eval
    return _convert(node_or_string)
  File "/usr/lib/python3.5/ast.py", line 83, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Compare object at 0x78172bee5358>


From marek.mosiewicz at jotel.com.pl  Sun Dec  9 07:01:35 2018
From: marek.mosiewicz at jotel.com.pl (Marek Mosiewicz)
Date: Sun, 09 Dec 2018 13:01:35 +0100
Subject: Creating type evaluation annotation
In-Reply-To: <slrnq0j09t.o84.jon+usenet@sable.unequivocal.eu>
References: <1544093325.1893.3.camel@jotel.com.pl>
 <mailman.85.1544116478.25456.python-list@python.org>
 <slrnq0j09t.o84.jon+usenet@sable.unequivocal.eu>
Message-ID: <1544356895.1323.2.camel@jotel.com.pl>

I'm talking about this https://docs.python.org/3/library/typing.html

I'm not talking about new language. I think it could be nice to have
standard PEP annotations for classes to make type validation or type
hints when writing code. 

It would be best of two worlds. Dynamic language with optional type
chceking if somebody wants it.?

In Groovy language there is annotation to indicate that you can not
make calls to methods which do not exist.

It could go futher and you could have annotation which validates
whatever method signature if it is valid. For example you can have
valiadator for something like Ruby on Rails in Ruby language for
methods which starts with find:
invoice.find_by_name
invoice.find_by_name_and_description

It could be even better and give you type hints when you type
method name. In this example when you write in IDE:
invoice.find_by_
it would suggest you futher possible method parts and possible
parameters for method.

This class valiadtors would be optional and run only when developer
wants to validate or write code, not in production as it is heavy.



W?dniu 06.12.2018, czw o?godzinie 20?06 +0000, u?ytkownik Jon Ribbens
napisa?:
> On 2018-12-06, Marek Mosiewicz <marek.mosiewicz at jotel.com.pl> wrote:
> > I'm Java developer,but had some experience with Python based
> > ERP software. It is quite serious application and I feel?
> > unconfortable with not having type checking. I do not say language
> > should be static, but having checking method signature
> > is big win. For example in refactoring.
> > I know that Python 3 has possibility to have indicate
> > type for varibale or param
> 
> Are you talking about http://www.mypy-lang.org/ ?


From brian_christians at hotmail.com  Sun Dec  9 11:30:13 2018
From: brian_christians at hotmail.com (Brian Christiansen)
Date: Sun, 9 Dec 2018 09:30:13 -0700
Subject: polar coordinates?
Message-ID: <pujfum$dsb$1@dont-email.me>

I have been messing with a program that is inspried by a video on 
youtube that is about the vizualization of pi.  I might make a post 
about that program someday, but I want to talk about something else. 
One of the ways of visualizing it is to put dots corresponding to each 
digits in a spiral pattern, in a color corresponding to what the digit 
is. I think this would be easiest, at least in the initial calculation 
of the point would be to use polar coordinates.

For example, if I were to use a very simple archimedian spiral, r = 0 + 
(1 x theta), the "first 4" points, if theta increases by 1 degree 
(2pi/360 radians), are (0,0) (2pi/360 "units",2pi/360"radians") 
(4pi/360, 4pi/360) (6pi/360,6pi/360).

The problem is that python (more specifically tkinter or graphics.py 
file that I downloaded) can't use polar coordinates directly to plot 
points (or at least I don't think they can). The polar coordinates have 
to be converted to cartesian coordinates, then converted to the 
coordinate system that a computer uses to actually plot points on the 
screen.

For example, the first point is just the origin, and its cartesian 
coordinates are just (0,0), but if I use the statement point(0,0) to 
plot the point, the computer will draw it at the upper left corner of 
the screen, not at the center where I want it to be, so I need to add 
however many pixels make up a "unit," to put it in the center of the 
screen.  For instance if my screen that I was drawing on is 700 by 700, 
I would have to add 350 to each coordinate value to get the point 
(350,350), then plotting on to that point would place the point where I 
want it, the center of the screen I have set up.

For the 2nd point, (2pi/360 "units", 2pi/360 radians) or (2pi/360 
"units", 1 degree), that converts to the cartesian coordinate ~(.0003 
"units",.0175 "units"), provided I did not get sin and cos mixed up, but 
then for the computer to actually plot it, the computer has to calculate 
how many actual pixels the x and y coordinates are offset from the 
origin, then add these values to the location on the computer where the 
origin is.  Since on the screen that was set up is 700 x 700 pixels 
(approximately the largest square screen I can put use on my computer), 
a "unit" is 350 pixels).

The X and Y offsets, recalcuated as pixels, is ~(350 x .0003,350 x 
.0175) or ~(.105,6.125).  Then these offsets added (or subtracted in the 
case of the y-coordinate since on a computer screen the y coordinates 
are upside down) to the actual location of the origin, (350,350), and 
rounded to the nearest integer, gives (350,344), which is the "screen 
coordinates" of the point that is actually plotted.

Basically what would happen is that a polar coordinate within a unit 
circle would be calculated, then the corresponding cartesian coordinate 
would be calculated from that, then the actual screen coordinates would 
be calculated based on where the center of the screen you set up is.

In setting up a screen, you might specify that the screen is 700 pixels 
by 700 pixels and a unit is 350 pixels.  By that mapping, any point 
outside of the unit circle (for the most part) will be off the screen. 
If a unit is defined as 175 pixels, that circle is increased to a radius 
of 2, if a unit is defined as 100 pixels, that radius is increased to 
3.5, if a unit is defined to be 50 pixels that radius is increased to 7. 
  I think defining a unit as even fewer pixels than that would lose too 
much resolution to be practical.

In sum, what all that would be is a method to map polar coordinates to 
actual screen coordinates.

I guess my question is if python can do this natively or if there is a 
package somewhere (polar.py?) that can do this. I know there are some 
functions in cmath? that can help with parts of this, but I don't think 
that python can do all of these things natively.  If there is a package 
(or include file) that can do all of these things, I have not been able 
to find it.

-- 
My Yonkoma: https://www.flickr.com/photos/brian0908/albums/72157680223526176

The E-mail associated with the account is a "spamcatcher" account that I 
got to every couple of months to empty out, and anything sent to it will 
not be seen for probably several months, if it is seen at all.
Brian Christiansen


From jon+usenet at unequivocal.eu  Sun Dec  9 11:37:10 2018
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Sun, 9 Dec 2018 16:37:10 -0000 (UTC)
Subject: Creating type evaluation annotation
References: <1544093325.1893.3.camel@jotel.com.pl>
 <mailman.85.1544116478.25456.python-list@python.org>
 <slrnq0j09t.o84.jon+usenet@sable.unequivocal.eu>
 <1544356895.1323.2.camel@jotel.com.pl>
 <mailman.10.1544359049.2771.python-list@python.org>
Message-ID: <slrnq0qh5m.6gl.jon+usenet@sable.unequivocal.eu>

On 2018-12-09, Marek Mosiewicz <marek.mosiewicz at jotel.com.pl> wrote:
> I'm talking about this https://docs.python.org/3/library/typing.html
>
> I'm not talking about new language. I think it could be nice to have
> standard PEP annotations for classes to make type validation or type
> hints when writing code. 
...
> This class valiadtors would be optional and run only when developer
> wants to validate or write code, not in production as it is heavy.

Yes, I think that's exactly what http://www.mypy-lang.org/ is.
It is not a new language, despite the rather strange choice of
domain name for it.


From cousinstanley at gmail.com  Sun Dec  9 12:07:54 2018
From: cousinstanley at gmail.com (Cousin Stanley)
Date: Sun, 09 Dec 2018 10:07:54 -0700
Subject: polar coordinates?
References: <pujfum$dsb$1@dont-email.me>
Message-ID: <puji5e$tnq$1@dont-email.me>

Brian Christiansen wrote:

> ....
> I guess my question is if python can do this natively 
> or if there is a package somewhere ( polar.py ? ) that can do this
> ....

  You might consider  matplotlib  for polar coordinate plotting ...

    https://matplotlib.org/examples/pylab_examples/polar_demo.html


-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona



From oscar.j.benjamin at gmail.com  Sun Dec  9 13:48:33 2018
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sun, 9 Dec 2018 18:48:33 +0000
Subject: polar coordinates?
In-Reply-To: <pujfum$dsb$1@dont-email.me>
References: <pujfum$dsb$1@dont-email.me>
Message-ID: <CAHVvXxQMB9-mUmsG6smM+LbLYom9gyWkf4MmKuQ_OKu_0d_ehw@mail.gmail.com>

On Sun, 9 Dec 2018 at 16:37, Brian Christiansen
<brian_christians at hotmail.com> wrote:
>
> I have been messing with a program that is inspried by a video on
> youtube that is about the vizualization of pi.  I might make a post
> about that program someday, but I want to talk about something else.
> One of the ways of visualizing it is to put dots corresponding to each
> digits in a spiral pattern, in a color corresponding to what the digit
> is. I think this would be easiest, at least in the initial calculation
> of the point would be to use polar coordinates.
>
> For example, if I were to use a very simple archimedian spiral, r = 0 +
> (1 x theta), the "first 4" points, if theta increases by 1 degree
> (2pi/360 radians), are (0,0) (2pi/360 "units",2pi/360"radians")
> (4pi/360, 4pi/360) (6pi/360,6pi/360).
>
> The problem is that python (more specifically tkinter or graphics.py
> file that I downloaded) can't use polar coordinates directly to plot
> points (or at least I don't think they can). The polar coordinates have
> to be converted to cartesian coordinates, then converted to the
> coordinate system that a computer uses to actually plot points on the
> screen.

Hi Brian,

I don't think anything exists (apart from matplotlib) to do this for you:
https://matplotlib.org/examples/pylab_examples/polar_demo.html

Converting from polar to Cartesian coordinates is easy enough though.
For your case if xc_p, yc_p are the pixel coordinates of the centre of
your window and rq and thetaq are the polar coordinates for point q
then

    from math import sin, cos
    xq_p = xc_p + r * cos(theta) * pix_scale
    yq_p = yc_p - r * sin(theta) * pix_scale

gives the pixel coordinates for q. The parameter pix_scale is the
number of pixels that corresponds to a distance of 1 in your polar
coordinate system. You might choose this parameter based on the
height/width in pixels of the window. Depending on what you're doing
you may need to convert xq_p and yq_p to int rounding in some way.

--
Oscar


From tjandacw at cox.net  Sun Dec  9 14:54:01 2018
From: tjandacw at cox.net (Tim Williams)
Date: Sun, 9 Dec 2018 11:54:01 -0800 (PST)
Subject: conda/anaconda and pip3 (pip)
In-Reply-To: <mailman.8.1544325181.2771.python-list@python.org>
References: <pu3po6$2t8$1@gioia.aioe.org> <pui13b$q1i$1@blaine.gmane.org>
 <mailman.8.1544325181.2771.python-list@python.org>
Message-ID: <0de9498f-0cc9-46bc-867c-0cecdd7ad0b0@googlegroups.com>

On Saturday, December 8, 2018 at 10:13:14 PM UTC-5, Monte Milanuk wrote:
> Did you find any solution(s)?

I usually just lurk and read on this list. I don't reply since there's usually more competent people that regularly post helpful answers. (I lurk to learn from them!)

If no one's replied yet, I'll give it my 2 cents ...

Without being a pip expert, I see from 'pip install -h' that you can specify where you want the package to be installed.

Install Options:
  -r, --requirement <file>    Install from the given requirements file. This option can be used multiple times.
  -c, --constraint <file>     Constrain versions using the given constraints file. This option can be used multiple
                              times.
  --no-deps                   Don't install package dependencies.
  --pre                       Include pre-release and development versions. By default, pip only finds stable
                              versions.
  -e, --editable <path/url>   Install a project in editable mode (i.e. setuptools "develop mode") from a local project
                              path or a VCS url.
  -t, --target <dir>          Install packages into <dir>. By default this will not replace existing files/folders in
                              <dir>. Use --upgrade to replace existing packages in <dir> with new versions.
  --user                      Install to the Python user install directory for your platform. Typically ~/.local/, or
                              %APPDATA%\Python on Windows. (See the Python documentation for site.USER_BASE for full
                              details.)
  --root <dir>                Install everything relative to this alternate root directory.
  --prefix <dir>              Installation prefix where lib, bin and other top-level folders are placed
  -b, --build <dir>           Directory to unpack packages into and build in. Note that an initial build still takes
                              place in a temporary directory. The location of temporary directories can be controlled
                              by setting the TMPDIR environment variable (TEMP on Windows) appropriately. When passed,
                              build directories are not cleaned in case of failures.
  --src <dir>                 Directory to check out editable projects into. The default in a virtualenv is "<venv
                              path>/src". The default for global installs is "<current dir>/src".
  -U, --upgrade               Upgrade all specified packages to the newest available version. The handling of
                              dependencies depends on the upgrade-strategy used.
  --upgrade-strategy <upgrade_strategy>
                              Determines how dependency upgrading should be handled [default: only-if-needed]. "eager"
                              - dependencies are upgraded regardless of whether the currently installed version
                              satisfies the requirements of the upgraded package(s). "only-if-needed" -  are upgraded
                              only when they do not satisfy the requirements of the upgraded package(s).
  --force-reinstall           Reinstall all packages even if they are already up-to-date.
  -I, --ignore-installed      Ignore the installed packages (reinstalling instead).
  --ignore-requires-python    Ignore the Requires-Python information.
  --no-build-isolation        Disable isolation when building a modern source distribution. Build dependencies
                              specified by PEP 518 must be already installed if this option is used.
  --install-option <options>  Extra arguments to be supplied to the setup.py install command (use like --install-
                              option="--install-scripts=/usr/local/bin"). Use multiple --install-option options to
                              pass multiple options to setup.py install. If you are using an option with a directory
                              path, be sure to use absolute path.
  --global-option <options>   Extra global options to be supplied to the setup.py call before the install command.
  --compile                   Compile Python source files to bytecode
  --no-compile                Do not compile Python source files to bytecode
  --no-warn-script-location   Do not warn when installing scripts outside PATH
  --no-warn-conflicts         Do not warn about broken dependencies
  --no-binary <format_control>
                              Do not use binary packages. Can be supplied multiple times, and each time adds to the
                              existing value. Accepts either :all: to disable all binary packages, :none: to empty the
                              set, or one or more package names with commas between them. Note that some packages are
                              tricky to compile and may fail to install when this option is used on them.
  --only-binary <format_control>
                              Do not use source packages. Can be supplied multiple times, and each time adds to the
                              existing value. Accepts either :all: to disable all source packages, :none: to empty the
                              set, or one or more package names with commas between them. Packages without binary
                              distributions will fail to install when this option is used on them.
  --no-clean                  Don't clean up build directories).
  --require-hashes            Require a hash to check each requirement against, for repeatable installs. This option
                              is implied when any package in a requirements file has a --hash option.
  --progress-bar <progress_bar>
                              Specify type of progress to be displayed [off|on|ascii|pretty|emoji] (default: on)


On Saturday, December 8, 2018 at 10:13:14 PM UTC-5, Monte Milanuk wrote:
> Did you find any solution(s)?



On Saturday, December 8, 2018 at 10:13:14 PM UTC-5, Monte Milanuk wrote:
> Did you find any solution(s)?



On Saturday, December 8, 2018 at 10:13:14 PM UTC-5, Monte Milanuk wrote:
> Did you find any solution(s)?

I'm thinking the the --target option may be the solution.

Again, just my 2 cents.


From __peter__ at web.de  Sun Dec  9 15:07:21 2018
From: __peter__ at web.de (Peter Otten)
Date: Sun, 09 Dec 2018 21:07:21 +0100
Subject: polar coordinates?
References: <pujfum$dsb$1@dont-email.me>
 <CAHVvXxQMB9-mUmsG6smM+LbLYom9gyWkf4MmKuQ_OKu_0d_ehw@mail.gmail.com>
Message-ID: <pujshm$sev$1@blaine.gmane.org>

Oscar Benjamin wrote:

> On Sun, 9 Dec 2018 at 16:37, Brian Christiansen
> <brian_christians at hotmail.com> wrote:
>>
>> I have been messing with a program that is inspried by a video on
>> youtube that is about the vizualization of pi.  I might make a post
>> about that program someday, but I want to talk about something else.
>> One of the ways of visualizing it is to put dots corresponding to each
>> digits in a spiral pattern, in a color corresponding to what the digit
>> is. I think this would be easiest, at least in the initial calculation
>> of the point would be to use polar coordinates.
>>
>> For example, if I were to use a very simple archimedian spiral, r = 0 +
>> (1 x theta), the "first 4" points, if theta increases by 1 degree
>> (2pi/360 radians), are (0,0) (2pi/360 "units",2pi/360"radians")
>> (4pi/360, 4pi/360) (6pi/360,6pi/360).
>>
>> The problem is that python (more specifically tkinter or graphics.py
>> file that I downloaded) can't use polar coordinates directly to plot
>> points (or at least I don't think they can). The polar coordinates have
>> to be converted to cartesian coordinates, then converted to the
>> coordinate system that a computer uses to actually plot points on the
>> screen.
> 
> Hi Brian,
> 
> I don't think anything exists (apart from matplotlib) to do this for you:
> https://matplotlib.org/examples/pylab_examples/polar_demo.html
> 
> Converting from polar to Cartesian coordinates is easy enough though.
> For your case if xc_p, yc_p are the pixel coordinates of the centre of
> your window and rq and thetaq are the polar coordinates for point q
> then
> 
>     from math import sin, cos
>     xq_p = xc_p + r * cos(theta) * pix_scale
>     yq_p = yc_p - r * sin(theta) * pix_scale
> 
> gives the pixel coordinates for q. The parameter pix_scale is the
> number of pixels that corresponds to a distance of 1 in your polar
> coordinate system. You might choose this parameter based on the
> height/width in pixels of the window. Depending on what you're doing
> you may need to convert xq_p and yq_p to int rounding in some way.

Python has native support for complex numbers. With these:

>>> def angle(deg):
...     return cmath.rect(1, math.radians(deg))
... 
>>> def point(c):
...     return (int(c.real), int(c.imag))
... 
>>> center = 350 + 350j
>>> p = center + 350
>>> point(p)
(700, 350)

To rotate p around the center by 90 degrees:

>>> point((p-center) * angle(90) + center)
(350, 700)

Scaling can also be performed with a single multiplication

>>> scale = 2 + 2j
>>> p * 2
(1400+700j)

Another option is to let the Canvas widget do it:

http://effbot.org/tkinterbook/canvas.htm#Tkinter.Canvas.scale-method




From tjreedy at udel.edu  Sun Dec  9 15:50:58 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 9 Dec 2018 15:50:58 -0500
Subject: Creating type evaluation annotation
In-Reply-To: <slrnq0qh5m.6gl.jon+usenet@sable.unequivocal.eu>
References: <1544093325.1893.3.camel@jotel.com.pl>
 <mailman.85.1544116478.25456.python-list@python.org>
 <slrnq0j09t.o84.jon+usenet@sable.unequivocal.eu>
 <1544356895.1323.2.camel@jotel.com.pl>
 <mailman.10.1544359049.2771.python-list@python.org>
 <slrnq0qh5m.6gl.jon+usenet@sable.unequivocal.eu>
Message-ID: <pujv3h$1as$1@blaine.gmane.org>

On 12/9/2018 11:37 AM, Jon Ribbens wrote:
> On 2018-12-09, Marek Mosiewicz <marek.mosiewicz at jotel.com.pl> wrote:
>> I'm talking about this https://docs.python.org/3/library/typing.html
>>
>> I'm not talking about new language. I think it could be nice to have
>> standard PEP annotations for classes to make type validation or type
>> hints when writing code.

>> This class valiadtors would be optional and run only when developer
>> wants to validate or write code, not in production as it is heavy.

> Yes, I think that's exactly what http://www.mypy-lang.org/ is.
> It is not a new language,

All libraries extend the language by adding names that python 
recognizes.  The typing additions are more like a new sublanguage than 
most in that the new objects are not for computation in the traditional 
sense but for representation of concepts.

Mypy recognizes this sublanguage to 'compute' whether calls conform to 
the defined interfaces.

> despite the rather strange choice of domain name for it.

mypy.org is a 'parked' domain name for sale.
mypy.com is a misc tech news site (nothing about Python) dead since Jan 
2014.

I don't know the genesis of the names above, but 'mypy' is a rather 
obvious contraction of 'my py'(thon).  I once had a directory of my 
python scripts called 'mypy', in a directory on sys.path, so I could run 
one with "python -m mypy.scriptname.  I changed my name to not conflict.

-- 
Terry Jan Reedy



From vlastimil.brom at gmail.com  Sun Dec  9 16:35:04 2018
From: vlastimil.brom at gmail.com (Vlastimil Brom)
Date: Sun, 9 Dec 2018 22:35:04 +0100
Subject: polar coordinates?
In-Reply-To: <pujfum$dsb$1@dont-email.me>
References: <pujfum$dsb$1@dont-email.me>
Message-ID: <CAHzaPEN+PG1XNwUEGUAyi6+osdGq54WNeOX7f6Sm_T5MsmSmhw@mail.gmail.com>

2018-12-09 17:30 GMT+01:00, Brian Christiansen <brian_christians at hotmail.com>:
> I have been messing with a program that is inspried by a video on
> youtube that is about the vizualization of pi.  I might make a post
> about that program someday, but I want to talk about something else.
> One of the ways of visualizing it is to put dots corresponding to each
> digits in a spiral pattern, in a color corresponding to what the digit
> is. I think this would be easiest, at least in the initial calculation
> of the point would be to use polar coordinates.
>
> For example, if I were to use a very simple archimedian spiral, r = 0 +
> (1 x theta), the "first 4" points, if theta increases by 1 degree
> (2pi/360 radians), are (0,0) (2pi/360 "units",2pi/360"radians")
> (4pi/360, 4pi/360) (6pi/360,6pi/360).
>
> The problem is that python (more specifically tkinter or graphics.py
> file that I downloaded) can't use polar coordinates directly to plot
> points (or at least I don't think they can).
>
[...]
> I guess my question is if python can do this natively or if there is a
> package somewhere (polar.py?) that can do this. I know there are some
> functions in cmath? that can help with parts of this, but I don't think
> that python can do all of these things natively.  If there is a package
> (or include file) that can do all of these things, I have not been able
> to find it.
>
[...]
>
> Brian Christiansen
> --
> https://mail.python.org/mailman/listinfo/python-list
>
Hi,
there were already some hints regarding the calculations;
I'd like to suggest some dedicated plotting library, if you have
special needs in this area.
E.g. matplotlib is very useful and well documented in my opinion,

cf.:
https://matplotlib.org/

for plotting using the polar projection:
https://matplotlib.org/gallery/pie_and_polar_charts/polar_demo.html#sphx-glr-gallery-pie-and-polar-charts-polar-demo-py

hth,
  vbr


From siddhesh at erpdata.cloud  Mon Dec 10 02:35:55 2018
From: siddhesh at erpdata.cloud (siddhesh at erpdata.cloud)
Date: Sun, 9 Dec 2018 23:35:55 -0800 (PST)
Subject: How to host python micro-services on windows machine?
Message-ID: <032596db-50f6-4875-ad94-54b446842e07@googlegroups.com>

I am developing some micro-services in python. I want to host that micro-services on windows machine. For ubuntu I am using Supervisord. What should I use for Windows ? Please help


From marek.mosiewicz at jotel.com.pl  Mon Dec 10 04:01:32 2018
From: marek.mosiewicz at jotel.com.pl (Marek Mosiewicz)
Date: Mon, 10 Dec 2018 10:01:32 +0100
Subject: Creating type evaluation annotation
In-Reply-To: <pujv3h$1as$1@blaine.gmane.org>
References: <1544093325.1893.3.camel@jotel.com.pl>
 <mailman.85.1544116478.25456.python-list@python.org>
 <slrnq0j09t.o84.jon+usenet@sable.unequivocal.eu>
 <1544356895.1323.2.camel@jotel.com.pl>
 <mailman.10.1544359049.2771.python-list@python.org>
 <slrnq0qh5m.6gl.jon+usenet@sable.unequivocal.eu>
 <pujv3h$1as$1@blaine.gmane.org>
Message-ID: <1544432492.1323.23.camel@jotel.com.pl>

W?dniu 09.12.2018, nie o?godzinie 15?50 -0500, u?ytkownik Terry Reedy
napisa?:
> 
> Mypy recognizes this sublanguage to 'compute' whether calls conform
> to?
> the defined interfaces.
> 
As far as I understand currently it computes if declaration commits to
static type. What I propose is to annotate type with "duck" typing
checker and futher it could make suggester for duck typing. We can
imagine duck typing suggester for imaginary entity class which accepts
finders like in Ruby on rails : "find_by_name" "find_by_id"
"find_by_name_and_something"

Then when you type "find_" it will suggest futher method name. When you
finish typing "find_by_name_and_length" for example it will force you
to put two paramters of type string and int
> 
> --?
> Terry Jan Reedy
> 


From a24061 at ducksburg.com  Mon Dec 10 04:25:59 2018
From: a24061 at ducksburg.com (Adam Funk)
Date: Mon, 10 Dec 2018 09:25:59 +0000
Subject: Basic pynomo instructions not working
References: <b5pdcfx7nb.ln2@news.ducksburg.com> <874lc9d4qz.fsf@handshake.de>
 <mailman.79.1542866879.2762.python-list@python.org>
Message-ID: <72o1efxdh4.ln2@news.ducksburg.com>

On 2018-11-22, dieter wrote:

> The "pynomo" version you have installed may have been developped for
> Python 2 and not run in "python3".
>
> In Python 2, you have implicit relative imports.
> As an example, it allows modules in the package "pynomo"
> to use "import nomo_wrapper" to import the submodule "nomo_wrapper".
> Python 3 has discarded implicit relative imports. In
> the example above, "import nomo_wrapper" must become
> "from . import nomo_wrapper" (explicit relative import)
> or "import pynomo.nomo_wrapper as nomo_wrapper" (absolute import).
>
> For the time being, you still find many packages which run
> only under Python 2. Failing relative imports or syntax errors
> are a frequent indication towards this.

Yup, that's the problem --- it does work in Python 2.  I didn't think
of that at first because there was a pip3 package for it!


-- 
Growth for growth's sake is the ideology of the cancer cell.
                                             ---Edward Abbey


From antoon.pardon at vub.be  Mon Dec 10 04:48:25 2018
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Mon, 10 Dec 2018 10:48:25 +0100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <CALwzidkekC9UUKdDdFb04UsVubu6udkFGfZXVJS1Q+5eqnj2Cg@mail.gmail.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <CAFDcVCRVrEB2Sp2Lhgn9kSUc=LiBw13Vja87H2sDMSCFpAd0dg@mail.gmail.com>
 <CALwzidkekC9UUKdDdFb04UsVubu6udkFGfZXVJS1Q+5eqnj2Cg@mail.gmail.com>
Message-ID: <ab8b207e-bcaf-bd79-61b1-c36eef584f88@vub.be>

On 8/12/18 09:35, Ian Kelly wrote:
> On Fri, Dec 7, 2018 at 11:56 PM Henrik Bengtsson
> <henrik.bengtsson at gmail.com> wrote:
>> A comment from the sideline: one could imagine extending the Python syntax
>> with a (optional) 0d prefix that allows for explicit specification of
>> decimal values. They would "complete" the family:
>>
>> * 0b: binary number
>> * 0o: octal number
>> * 0d: decimal number
>> * 0x: hexadecimal number
> That's nice and elegant, but what would be the use case?

A use case is that sometimes numbers are a code for something else and this
something else is more recognizable if all such coded numbers are written in
a common length. The normal way to write such numbers in a common length
is to start the number with sufficient zeroes. But that can now only
be done in binary, octal and hexadecimal notation.

Antoon.



From antoon.pardon at vub.be  Mon Dec 10 05:00:20 2018
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Mon, 10 Dec 2018 11:00:20 +0100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <20181208050002.GA62588@cskk.homeip.net>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
Message-ID: <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>

On 8/12/18 06:00, Cameron Simpson wrote:
> On 07Dec2018 20:24, Jach Fong <jfong at ms4.hinet.net> wrote:
>> Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
>>> What is it exactly that you're trying to accomplish with this? Perhaps
>>> there's a better way than using eval.
>>
>> This problem comes from solving a word puzzle,
>> ?? ab + aa + cd == ce
>> Each character will be translate to a digit and evaluate the
>> correctness,
>> ?? 03 + 00 + 15 == 18
>
> Then you should be evaluating the digits and assembling values from
> them. Not trying to shoehorn a string through something that _might_
> accept this string and do what you want. In Python 2 it will accept
> your string and not do what you want; at least in Python 3 it doesn't
> accept your string.
>
> My point here is that the structure of your puzzle doesn't map
> directly into a naive python statement, and you shouldn't be
> pretending it might.

How do you figure? As far as I understand he is trying to solve this kind of puzzle:

    SEND
    MORE
 +  ????
   MONEY

Where every letter is to be replaced by a digit in such a way that the sum checks out. Sure trying to
solve this by starting with the string: "SEND + MORE == MONEY" and doing replaces until eval comes up
with true is not very sofisticated but I wouldn't call it shoehorning either.

-- 
Antoon.



From rosuav at gmail.com  Mon Dec 10 05:03:35 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 10 Dec 2018 21:03:35 +1100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
Message-ID: <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>

On Mon, Dec 10, 2018 at 9:01 PM Antoon Pardon <antoon.pardon at vub.be> wrote:
>
> On 8/12/18 06:00, Cameron Simpson wrote:
> > On 07Dec2018 20:24, Jach Fong <jfong at ms4.hinet.net> wrote:
> >> Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
> >>> What is it exactly that you're trying to accomplish with this? Perhaps
> >>> there's a better way than using eval.
> >>
> >> This problem comes from solving a word puzzle,
> >>    ab + aa + cd == ce
> >> Each character will be translate to a digit and evaluate the
> >> correctness,
> >>    03 + 00 + 15 == 18
> >
> > Then you should be evaluating the digits and assembling values from
> > them. Not trying to shoehorn a string through something that _might_
> > accept this string and do what you want. In Python 2 it will accept
> > your string and not do what you want; at least in Python 3 it doesn't
> > accept your string.
> >
> > My point here is that the structure of your puzzle doesn't map
> > directly into a naive python statement, and you shouldn't be
> > pretending it might.
>
> How do you figure? As far as I understand he is trying to solve this kind of puzzle:
>
>     SEND
>     MORE
>  +  ????
>    MONEY
>
> Where every letter is to be replaced by a digit in such a way that the sum checks out. Sure trying to
> solve this by starting with the string: "SEND + MORE == MONEY" and doing replaces until eval comes up
> with true is not very sofisticated but I wouldn't call it shoehorning either.
>

Considering that, in a problem of that description, neither S nor M
may represent zero, I don't think there's a problem here.

ChrisA


From antoon.pardon at vub.be  Mon Dec 10 05:07:11 2018
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Mon, 10 Dec 2018 11:07:11 +0100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <pufpvh$1pt$1@blaine.gmane.org>
References: <76bedc56-55af-4523-bdef-a41b716b1d37@googlegroups.com>
 <pufpvh$1pt$1@blaine.gmane.org>
Message-ID: <76060b89-a6e4-14bb-e1cc-dedc4997da7b@vub.be>

On 8/12/18 07:59, Serhiy Storchaka wrote:
> 08.12.18 03:17, jfong at ms4.hinet.net ????:
>>>>> 00
>> 0
>>>>> 03
>> ?? File "<stdin>", line 1
>> ???? 03
>> ????? ^
>> SyntaxError: invalid token
>
>
> In Python 3.8 the error message will be more informative:
>
>>>> 03
> ? File "<stdin>", line 1
> SyntaxError: leading zeros in decimal integer literals are not
> permitted; use an 0o prefix for octal integers
>
That is not helpful if it makes the code more meaningful to write 073
and I want it to have the value 73 not 0o73. -- Antoon.



From antoon.pardon at vub.be  Mon Dec 10 05:09:58 2018
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Mon, 10 Dec 2018 11:09:58 +0100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
 <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>
Message-ID: <a5b078fa-fe04-2f11-7bd4-5a834b83bbf5@vub.be>

On 10/12/18 11:03, Chris Angelico wrote:
> On Mon, Dec 10, 2018 at 9:01 PM Antoon Pardon <antoon.pardon at vub.be> wrote:
>> On 8/12/18 06:00, Cameron Simpson wrote:
>>> On 07Dec2018 20:24, Jach Fong <jfong at ms4.hinet.net> wrote:
>>>> Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
>>>>> What is it exactly that you're trying to accomplish with this? Perhaps
>>>>> there's a better way than using eval.
>>>> This problem comes from solving a word puzzle,
>>>>    ab + aa + cd == ce
>>>> Each character will be translate to a digit and evaluate the
>>>> correctness,
>>>>    03 + 00 + 15 == 18
>>> Then you should be evaluating the digits and assembling values from
>>> them. Not trying to shoehorn a string through something that _might_
>>> accept this string and do what you want. In Python 2 it will accept
>>> your string and not do what you want; at least in Python 3 it doesn't
>>> accept your string.
>>>
>>> My point here is that the structure of your puzzle doesn't map
>>> directly into a naive python statement, and you shouldn't be
>>> pretending it might.
>> How do you figure? As far as I understand he is trying to solve this kind of puzzle:
>>
>>     SEND
>>     MORE
>>  +  ????
>>    MONEY
>>
>> Where every letter is to be replaced by a digit in such a way that the sum checks out. Sure trying to
>> solve this by starting with the string: "SEND + MORE == MONEY" and doing replaces until eval comes up
>> with true is not very sofisticated but I wouldn't call it shoehorning either.
>>
> Considering that, in a problem of that description, neither S nor M
> may represent zero, I don't think there's a problem here.

Not all such problems have that condition.

-- 
Antoon.



From rosuav at gmail.com  Mon Dec 10 05:16:46 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 10 Dec 2018 21:16:46 +1100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <a5b078fa-fe04-2f11-7bd4-5a834b83bbf5@vub.be>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
 <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>
 <a5b078fa-fe04-2f11-7bd4-5a834b83bbf5@vub.be>
Message-ID: <CAPTjJmo3j4r75UjF0s+0-NjNcMXcx5SA=SqsAfaTJx5-dX4Bzg@mail.gmail.com>

On Mon, Dec 10, 2018 at 9:11 PM Antoon Pardon <antoon.pardon at vub.be> wrote:
>
> On 10/12/18 11:03, Chris Angelico wrote:
> > Considering that, in a problem of that description, neither S nor M
> > may represent zero, I don't think there's a problem here.
>
> Not all such problems have that condition.

They should. Every published set of problems that I've ever solved by
hand has. I went searching online for some, and found this page:

http://puzzlepicnic.com/genre?alphametic

which clearly states that exact restriction. The implication is that
you're solving a puzzle in arithmetic (usually addition or long
multiplication), and it is *exactly* as you would have written it with
digits, save that the digits have been replaced with letters (and
carries have been omitted, since that'd make it too easy). You
wouldn't write a leading zero on a number in standard grade-school
arithmetic, so you also won't use a leading zero in anything here.

ChrisA


From jfong at ms4.hinet.net  Mon Dec 10 05:40:59 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Mon, 10 Dec 2018 02:40:59 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <mailman.22.1544437021.2771.python-list@python.org>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
 <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>
 <a5b078fa-fe04-2f11-7bd4-5a834b83bbf5@vub.be>
 <CAPTjJmo3j4r75UjF0s+0-NjNcMXcx5SA=SqsAfaTJx5-dX4Bzg@mail.gmail.com>
 <mailman.22.1544437021.2771.python-list@python.org>
Message-ID: <9e8263f4-9e08-40ba-841d-5aaa776f24ce@googlegroups.com>

Chris Angelico? 2018?12?10???? UTC+8??6?17?14????
> On Mon, Dec 10, 2018 at 9:11 PM Antoon Pardon <antoon.pardon at vub.be> wrote:
> >
> > On 10/12/18 11:03, Chris Angelico wrote:
> > > Considering that, in a problem of that description, neither S nor M
> > > may represent zero, I don't think there's a problem here.
> >
> > Not all such problems have that condition.
> 
> They should. Every published set of problems that I've ever solved by
> hand has. I went searching online for some, and found this page:
> 
> http://puzzlepicnic.com/genre?alphametic
> 
> which clearly states that exact restriction. The implication is that
> you're solving a puzzle in arithmetic (usually addition or long
> multiplication), and it is *exactly* as you would have written it with
> digits, save that the digits have been replaced with letters (and
> carries have been omitted, since that'd make it too easy). You
> wouldn't write a leading zero on a number in standard grade-school
> arithmetic, so you also won't use a leading zero in anything here.
> 
> ChrisA

All I know is that when I write a number 03, there is no any human being will say it's an illegal number.

I prefer to buy the reason that this restriction was bring in is because of the puzzle's author know it will cause trouble without this, not because of our written habit.

--Jach


From rosuav at gmail.com  Mon Dec 10 06:08:30 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 10 Dec 2018 22:08:30 +1100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <9e8263f4-9e08-40ba-841d-5aaa776f24ce@googlegroups.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
 <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>
 <a5b078fa-fe04-2f11-7bd4-5a834b83bbf5@vub.be>
 <CAPTjJmo3j4r75UjF0s+0-NjNcMXcx5SA=SqsAfaTJx5-dX4Bzg@mail.gmail.com>
 <mailman.22.1544437021.2771.python-list@python.org>
 <9e8263f4-9e08-40ba-841d-5aaa776f24ce@googlegroups.com>
Message-ID: <CAPTjJmp1b0PK5o7K4vncO9HoWTSmXzMwF8z5xqS_z=vu077ehA@mail.gmail.com>

On Mon, Dec 10, 2018 at 9:46 PM <jfong at ms4.hinet.net> wrote:
>
> Chris Angelico? 2018?12?10???? UTC+8??6?17?14????
> > On Mon, Dec 10, 2018 at 9:11 PM Antoon Pardon <antoon.pardon at vub.be> wrote:
> > >
> > > On 10/12/18 11:03, Chris Angelico wrote:
> > > > Considering that, in a problem of that description, neither S nor M
> > > > may represent zero, I don't think there's a problem here.
> > >
> > > Not all such problems have that condition.
> >
> > They should. Every published set of problems that I've ever solved by
> > hand has. I went searching online for some, and found this page:
> >
> > http://puzzlepicnic.com/genre?alphametic
> >
> > which clearly states that exact restriction. The implication is that
> > you're solving a puzzle in arithmetic (usually addition or long
> > multiplication), and it is *exactly* as you would have written it with
> > digits, save that the digits have been replaced with letters (and
> > carries have been omitted, since that'd make it too easy). You
> > wouldn't write a leading zero on a number in standard grade-school
> > arithmetic, so you also won't use a leading zero in anything here.
> >
> > ChrisA
>
> All I know is that when I write a number 03, there is no any human being will say it's an illegal number.
>

Yet most human beings will agree that you don't write out an
arithmetic problem as:

   0 1 9 8
 +   7 1 3
 =========

> I prefer to buy the reason that this restriction was bring in is because of the puzzle's author know it will cause trouble without this, not because of our written habit.
>

No, it's a restriction because it is unsatisfactory without it. The
point of a puzzle is to be fun, and fun means having restrictions that
fit what people expect.

ChrisA


From jfong at ms4.hinet.net  Mon Dec 10 06:28:37 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Mon, 10 Dec 2018 03:28:37 -0800 (PST)
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <mailman.23.1544440124.2771.python-list@python.org>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
 <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>
 <a5b078fa-fe04-2f11-7bd4-5a834b83bbf5@vub.be>
 <CAPTjJmo3j4r75UjF0s+0-NjNcMXcx5SA=SqsAfaTJx5-dX4Bzg@mail.gmail.com>
 <mailman.22.1544437021.2771.python-list@python.org>
 <9e8263f4-9e08-40ba-841d-5aaa776f24ce@googlegroups.com>
 <CAPTjJmp1b0PK5o7K4vncO9HoWTSmXzMwF8z5xqS_z=vu077ehA@mail.gmail.com>
 <mailman.23.1544440124.2771.python-list@python.org>
Message-ID: <31e94b2c-ca14-4a4b-8c4d-39bf35c2e20a@googlegroups.com>

Chris Angelico? 2018?12?10???? UTC+8??7?09?01????
> On Mon, Dec 10, 2018 at 9:46 PM <jfong at ms4.hinet.net> wrote:
> >
> > Chris Angelico? 2018?12?10???? UTC+8??6?17?14????
> > > On Mon, Dec 10, 2018 at 9:11 PM Antoon Pardon <antoon.pardon at vub.be> wrote:
> > > >
> > > > On 10/12/18 11:03, Chris Angelico wrote:
> > > > > Considering that, in a problem of that description, neither S nor M
> > > > > may represent zero, I don't think there's a problem here.
> > > >
> > > > Not all such problems have that condition.
> > >
> > > They should. Every published set of problems that I've ever solved by
> > > hand has. I went searching online for some, and found this page:
> > >
> > > http://puzzlepicnic.com/genre?alphametic
> > >
> > > which clearly states that exact restriction. The implication is that
> > > you're solving a puzzle in arithmetic (usually addition or long
> > > multiplication), and it is *exactly* as you would have written it with
> > > digits, save that the digits have been replaced with letters (and
> > > carries have been omitted, since that'd make it too easy). You
> > > wouldn't write a leading zero on a number in standard grade-school
> > > arithmetic, so you also won't use a leading zero in anything here.
> > >
> > > ChrisA
> >
> > All I know is that when I write a number 03, there is no any human being will say it's an illegal number.
> >
> 
> Yet most human beings will agree that you don't write out an
> arithmetic problem as:
> 
>    0 1 9 8
>  +   7 1 3
>  =========

Python3 gives me the error message is because of the number 0198, not because of 0198 + 713.

> > I prefer to buy the reason that this restriction was bring in is because of the puzzle's author know it will cause trouble without this, not because of our written habit.
> >
> 
> No, it's a restriction because it is unsatisfactory without it. The
> point of a puzzle is to be fun, and fun means having restrictions that
> fit what people expect.

The fun is from solving the puzzle, not from its restriction, unless the puzzle has no fun without this restriction.

--Jach

> ChrisA



From rosuav at gmail.com  Mon Dec 10 06:38:52 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 10 Dec 2018 22:38:52 +1100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <31e94b2c-ca14-4a4b-8c4d-39bf35c2e20a@googlegroups.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
 <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>
 <a5b078fa-fe04-2f11-7bd4-5a834b83bbf5@vub.be>
 <CAPTjJmo3j4r75UjF0s+0-NjNcMXcx5SA=SqsAfaTJx5-dX4Bzg@mail.gmail.com>
 <mailman.22.1544437021.2771.python-list@python.org>
 <9e8263f4-9e08-40ba-841d-5aaa776f24ce@googlegroups.com>
 <CAPTjJmp1b0PK5o7K4vncO9HoWTSmXzMwF8z5xqS_z=vu077ehA@mail.gmail.com>
 <mailman.23.1544440124.2771.python-list@python.org>
 <31e94b2c-ca14-4a4b-8c4d-39bf35c2e20a@googlegroups.com>
Message-ID: <CAPTjJmp6U1-S5u64ntcQ0PSnbPrEeeiqecqAaD-ehbQgyjv0Pg@mail.gmail.com>

On Mon, Dec 10, 2018 at 10:31 PM <jfong at ms4.hinet.net> wrote:
>
> Chris Angelico? 2018?12?10???? UTC+8??7?09?01????
> > Yet most human beings will agree that you don't write out an
> > arithmetic problem as:
> >
> >    0 1 9 8
> >  +   7 1 3
> >  =========
>
> Python3 gives me the error message is because of the number 0198, not because of 0198 + 713.

As would humans, yes. You don't write numbers with leading zeros in
normal grade-school arithmetic. Or at least, I was taught not to.
(Except in special circumstances.)

> > > I prefer to buy the reason that this restriction was bring in is because of the puzzle's author know it will cause trouble without this, not because of our written habit.
> > >
> >
> > No, it's a restriction because it is unsatisfactory without it. The
> > point of a puzzle is to be fun, and fun means having restrictions that
> > fit what people expect.
>
> The fun is from solving the puzzle, not from its restriction, unless the puzzle has no fun without this restriction.
>

Okay. Here, solve this one.

send + more = digits

There are absolutely no restrictions. A letter could represent the
square root of negative one, or the base of natural logarithms, or
anything at all. Is that fun? Or would you prefer to have normal and
sane restrictions?

Restrictions are there to create fun. Restrictions make it into an
interesting puzzle instead of a "gotcha" that makes you feel stupid
when someone points it out (for instance, you say "there's no
solution" and someone says "ah ha, but you see, the letter 's'
represents the colour green, not a digit").

In terms of game/puzzle design, the restriction is a good one. In
terms of programming language design, it's also a good restriction,
though for different reasons (mainly the conflict between the
C-derived expectation and the human-derived expectation, and the
subtle confusion that would be caused).

ChrisA


From oliver.schoenborn at gmail.com  Mon Dec 10 07:17:45 2018
From: oliver.schoenborn at gmail.com (oliver)
Date: Mon, 10 Dec 2018 07:17:45 -0500
Subject: How to host python micro-services on windows machine?
In-Reply-To: <032596db-50f6-4875-ad94-54b446842e07@googlegroups.com>
References: <032596db-50f6-4875-ad94-54b446842e07@googlegroups.com>
Message-ID: <CAMb+DaU9DzkF9Kp-7w1PQ85568cLG0bWy-FLPRzFnUyY1kXbrw@mail.gmail.com>

There are a few options in https://stackoverflow.com/q/7629813/869951, esp
the third answer.

On Mon., Dec. 10, 2018, 02:41 , <siddhesh at erpdata.cloud> wrote:

> I am developing some micro-services in python. I want to host that
> micro-services on windows machine. For ubuntu I am using Supervisord. What
> should I use for Windows ? Please help
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
Oliver Schoenborn
  || Cloud Application Engineer, Owner || Sentian Software Engineering ||
Ottawa, ON, Canada
  || +1-613-552-4466 (mobile) || @schollii2 (Twitter) || schoenborno (Skype)
  || LinkedIn <https://linkedin.com/in/oliver-schoenborn> || StackOverFlow
<https://stackoverflow.com/users/869951/oliver> || CodeProject
<https://www.codeproject.com/Articles/schollii#Tip> || GitHub
<https://github.com/schollii>


From oliver.schoenborn at gmail.com  Mon Dec 10 07:54:45 2018
From: oliver.schoenborn at gmail.com (oliver)
Date: Mon, 10 Dec 2018 07:54:45 -0500
Subject: How to host python micro-services on windows machine?
In-Reply-To: <CAMb+DaU9DzkF9Kp-7w1PQ85568cLG0bWy-FLPRzFnUyY1kXbrw@mail.gmail.com>
References: <032596db-50f6-4875-ad94-54b446842e07@googlegroups.com>
 <CAMb+DaU9DzkF9Kp-7w1PQ85568cLG0bWy-FLPRzFnUyY1kXbrw@mail.gmail.com>
Message-ID: <CAMb+DaV2KUOMxGezFt-Wwi5GVCJbO4y-L90kdviFYSpR+FjTjg@mail.gmail.com>

You might also find that
https://www.raymond.cc/blog/keep-application-running-by-automatically-rerun-when-closed/
lists some useful options.

But most solutions I read about for Windows seem to be based on its
Services subsystem and its ability to restart and control how many time to
retry.

On Mon., Dec. 10, 2018, 07:17 oliver, <oliver.schoenborn at gmail.com> wrote:

> There are a few options in https://stackoverflow.com/q/7629813/869951,
> esp the third answer.
>
> On Mon., Dec. 10, 2018, 02:41 , <siddhesh at erpdata.cloud> wrote:
>
>> I am developing some micro-services in python. I want to host that
>> micro-services on windows machine. For ubuntu I am using Supervisord. What
>> should I use for Windows ? Please help
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
> --
> Oliver Schoenborn
>   || Cloud Application Engineer, Owner || Sentian Software Engineering ||
> Ottawa, ON, Canada
>   || +1-613-552-4466 (mobile) || @schollii2 (Twitter) || schoenborno (
> Skype)
>   || LinkedIn <https://linkedin.com/in/oliver-schoenborn> || StackOverFlow
> <https://stackoverflow.com/users/869951/oliver> || CodeProject
> <https://www.codeproject.com/Articles/schollii#Tip> || GitHub
> <https://github.com/schollii>
>
> --
Oliver Schoenborn
  || Cloud Application Engineer, Owner || Sentian Software Engineering ||
Ottawa, ON, Canada
  || +1-613-552-4466 (mobile) || @schollii2 (Twitter) || schoenborno (Skype)
  || LinkedIn <https://linkedin.com/in/oliver-schoenborn> || StackOverFlow
<https://stackoverflow.com/users/869951/oliver> || CodeProject
<https://www.codeproject.com/Articles/schollii#Tip> || GitHub
<https://github.com/schollii>


From antoon.pardon at vub.be  Mon Dec 10 08:10:55 2018
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Mon, 10 Dec 2018 14:10:55 +0100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <CAPTjJmo3j4r75UjF0s+0-NjNcMXcx5SA=SqsAfaTJx5-dX4Bzg@mail.gmail.com>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
 <CAPTjJmpTdqQ5_bLGsM+wFbB=V4VYPZYa3-B0yK9F+8C-CoDPww@mail.gmail.com>
 <a5b078fa-fe04-2f11-7bd4-5a834b83bbf5@vub.be>
 <CAPTjJmo3j4r75UjF0s+0-NjNcMXcx5SA=SqsAfaTJx5-dX4Bzg@mail.gmail.com>
Message-ID: <8605aa27-c358-08be-a710-39ba8156c445@vub.be>

On 10/12/18 11:16, Chris Angelico wrote:
> On Mon, Dec 10, 2018 at 9:11 PM Antoon Pardon <antoon.pardon at vub.be> wrote:
>> On 10/12/18 11:03, Chris Angelico wrote:
>>> Considering that, in a problem of that description, neither S nor M
>>> may represent zero, I don't think there's a problem here.
>> Not all such problems have that condition.
> They should. Every published set of problems that I've ever solved by
> hand has. I went searching online for some, and found this page:
>
> http://puzzlepicnic.com/genre?alphametic
>
> which clearly states that exact restriction.

Why should they? Is this some holy restriction a puzzle maker has
to follow on pain of excommunication somehow? Not all replacement
puzzles are alphametic puzzles.

-- 
Antoon.




From lizhollinshead5 at gmail.com  Mon Dec 10 14:26:03 2018
From: lizhollinshead5 at gmail.com (lizhollinshead5 at gmail.com)
Date: Mon, 10 Dec 2018 11:26:03 -0800 (PST)
Subject: tkinter resizable text with grid
In-Reply-To: <puahie$10mt$1@gioia.aioe.org>
References: <puahie$10mt$1@gioia.aioe.org>
Message-ID: <5791bdb5-b73e-443e-8d51-82bfe67bf4df@googlegroups.com>

On Thursday, 6 December 2018 07:02:50 UTC, Paulo da Silva  wrote:
> Hi!
> 
> Does anybody know why this code does not expand the text widget when I
> increase the window size (with mouse)? I want height and width but as
> minimum (or may be initial) size.
> 
> import tkinter as tk
> 
> class App:
> 	def __init__(self,master):
> 		self.tboard=tk.Text(master,height=40,width=50)
> 		self.tboard.grid(row=1,column=1,sticky="nsew")
> 		self.tboard.grid_rowconfigure(1,weight=1)
> 		self.tboard.grid_columnconfigure(1,weight=1)
> 
> root=tk.Tk()
> app=App(root)
> 
> root.mainloop()
> 
> Thanks

Others here have commented about Tkinter.  I'm not a professional programmer, and I struggled with Python2 and Tkinter for a while.  It worked, but it was a struggle.
*
A few years ago I started using Glade and Python3 (and gi.repository).  Much easier, much less Python code, much easier to maintain.
*
Not that I'm much of a critic of Tkinter, just that the alternative is simpler and easier.


From avigross at verizon.net  Mon Dec 10 21:11:03 2018
From: avigross at verizon.net (Avi Gross)
Date: Mon, 10 Dec 2018 21:11:03 -0500
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
Message-ID: <009e01d490f6$c4c44a60$4e4cdf20$@verizon.net>

READERS DIGEST CONDENSED QUESTION: How expensive is eval("123 + 456 == 975") versus other ways?

The discussion started by Jach has spread away from discussing how python deals with numbers starting with leading zeroes such as "03". I note there are many ID numbers like social security that have a leading zero so if converted to an int for storage reasons, ...

The TOPIC I want to discuss is the concerns about Jach wanting to use "eval()". Besides security concerns, I want to know if it is particularly expensive.

It was suggested that you might solve a range of problems like puzzles asking you to substitute unique digits for letters. Some of  these puzzles are  normally solved with logic a step at a time but the techniques used may require a specific order and may not be easy to program for a more general case so can it be solved using what might be termed brute force. I mean try EVERYTHING that might work, including some that might not.

A particular example was: SEND + MORE = MONEY.

Jach's suggestion was to take every possible combination of digits and make the substitutions for the letters in that puzzle: then do something like this:

>>> eval ("9567+1085==10652")
True
>>> eval("123+456==975")
False

So, although better algorithms exist, no doubt, I considered what it would take to do this without an eval. I came up with the following as an outline.

Accept the puzzle as a string with any number of items to the right or left of an "=" as long as they only have whitespace and plus signs in addition to alphabetic characters. So "AE + DF = CE + AB + AA" would be fine. This model does not include doing subtraction or other things, just a simple model like that, for now.

You can easily remove whitespace, force everything to one case, split it into a left/right on "=" and then split those into lists on "+.

You now have two lists of alphabetic strings that ultimately must become lists of integers and then the sums can be compared.

To get the number of unique letters in the puzzle, N, you throw all the list items as individual characters into a set. Clearly, for this kind of puzzle, there cannot be more than ten unique letters or there can be no unique mapping for 0-9.

I would then "generate" all possible combinations of digits 0-9 of length N. There are an amazing number of ways to do that ranging from taking a range(10**N) and converting it to a string then a list of numeral characters then tossing out any that have any duplicates, to creating a recursive function that passes along what has been used so far to the next call. Since many of these problems only need ONE solution, the generator would only iterate as many times as needed and no giant lists of numbers or strings need be in memory at any time.

For each tuple returned  by the generator, you can iterate on the set of unique letters and use string functions to substitute the digits, or perhaps do this all at once. You would do this to all the items in what I call the left list as well as all the items on the right list. These would not be "numeric" so using int() on each item  would work EVEN with leading zeroes. Seems safe enough.

Finally, with no eval needed, you take the sum of the numbers in a list of the left and compare to the sum on the right and if equal, you present the solution in whatever way works. If no more solutions are needed, you quit. I might write this part as a generator too, that can be called once or to completion.

Yes, this is brute force. Using range(1,000,000) (no commas in real use) would be a million iterations when there are six unique characters in the puzzle and as many as 10 billion if all ten are in use. If you use nested loops like I showed a while ago (albeit to make arbitrary ones for many sizes could require multiple functions or use of an eval on one built by hand) you can cut down the number of iterations as the nested loops count down with each one doing one less than the one before it. Same goes for the recursive function call method as it passes along what numerals  have already been used. There may already be a permutation function that does this efficiently in C.

The above description includes parts that may also be used in the eval situation. The main difference may be that my method uses int() and perhaps some string functions. So, does eval just divert the attention of the interpreter as may happen with importing a module, or does it invoke a second copy of the interpreter as may happen with some multitasking methods? I would guess the former. But it may also have to first compile the text into byte code. However, a full eval is like using a sledgehammer when a thumbtack might be enough. Then again, it is already sitting there so a little extra use might be cheap.

So a main reason for my variant might still be to avoid taking any chances on rogue code. Mind you, in the above I would remove everything except [a-zA-Z=+] including parentheses and periods so damage from a carefully crafted text should be limited to over-riding a global variable name and even that can be handled by supplying a minimal environment to the eval call.

The real cost that dominates here is not memory, albeit garbage collection may be busy as it generates lots of temporary small bits of data. It is how the number of iterations grows.

I have looked at a somewhat related issue of how you generate all possible SCRABBLE words (or BOGGLE or ...) given specific starting letters. That problem allows duplicate letters but it has to handle quite large words (even weird medical terms like Pneumonoultramicroscopicsilicovolcanoconiosis but SCRABBLE words cannot possible be larger than the seven letters you have added to whatever is on the board that it connects to and in any case, the board can only fit 19 letters.) One way to make all possible combinations is along the lines above with many needed changes as there can (in theory) be as many as 26 unique letters (in English) and you can have multiple copies of some letters. If you allow other languages, the problem grows to the point where brute force is not realistic. And, ideally, you winnow down your choices by checking each word against a large dictionary. Anyone know of one that has a decent selection and can be freely imported? I mean one word per line.

Of course, in this case, there is room for improvement such as by making a tree out of the dictionary and pruning any attempts if the word you are building wanders off a leaf. Another story.

I apologize for the length. The main question was whether eval is particularly expensive.

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Antoon Pardon
Sent: Monday, December 10, 2018 5:00 AM
To: python-list at python.org
Subject: Re: Why Python don't accept 03 as a number?

On 8/12/18 06:00, Cameron Simpson wrote:
> On 07Dec2018 20:24, Jach Fong <jfong at ms4.hinet.net> wrote:
>> Ian at 2018/12/8 UTC+8 AM11:28:34 wrote:
>>> What is it exactly that you're trying to accomplish with this? 
>>> Perhaps there's a better way than using eval.
>>
>> This problem comes from solving a word puzzle,
>>    ab + aa + cd == ce
>> Each character will be translate to a digit and evaluate the 
>> correctness,
>>    03 + 00 + 15 == 18
>
> Then you should be evaluating the digits and assembling values from 
> them. Not trying to shoehorn a string through something that _might_ 
> accept this string and do what you want. In Python 2 it will accept 
> your string and not do what you want; at least in Python 3 it doesn't 
> accept your string.
>
> My point here is that the structure of your puzzle doesn't map 
> directly into a naive python statement, and you shouldn't be 
> pretending it might.

How do you figure? As far as I understand he is trying to solve this kind of puzzle:

    SEND
    MORE
 +  ????
   MONEY

Where every letter is to be replaced by a digit in such a way that the sum checks out. Sure trying to solve this by starting with the string: "SEND + MORE == MONEY" and doing replaces until eval comes up with true is not very sofisticated but I wouldn't call it shoehorning either.

--
Antoon.

--
https://mail.python.org/mailman/listinfo/python-list



From rosuav at gmail.com  Mon Dec 10 21:46:25 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 11 Dec 2018 13:46:25 +1100
Subject: Why Python don't accept 03 as a number?
In-Reply-To: <009e01d490f6$c4c44a60$4e4cdf20$@verizon.net>
References: <21d8cf6a-735f-4d94-86ef-c3d7149b8050@googlegroups.com>
 <20181208050002.GA62588@cskk.homeip.net>
 <8755c86b-8144-5607-13a0-f9fa9c6f0bbe@vub.be>
 <009e01d490f6$c4c44a60$4e4cdf20$@verizon.net>
Message-ID: <CAPTjJmqi0ECj63xmM5BTZ93LMZ50zpEqqLpbeF76KQvr_bBiEQ@mail.gmail.com>

On Tue, Dec 11, 2018 at 1:12 PM Avi Gross <avigross at verizon.net> wrote:
> I would then "generate" all possible combinations of digits 0-9 of length N. There are an amazing number of ways to do that ranging from taking a range(10**N) and converting it to a string then a list of numeral characters then tossing out any that have any duplicates, to creating a recursive function that passes along what has been used so far to the next call. Since many of these problems only need ONE solution, the generator would only iterate as many times as needed and no giant lists of numbers or strings need be in memory at any time.
>

First off, you can slash that number significantly, as there are only
10! (not 10**10) possibilities. Secondly, you can eliminate zero from
consideration from any letter that starts a sequence other than a
single digit, as demonstrated previously. So there are 9! * x, where x
is the number of letters that could potentially be zero. That cuts
your total possibilities from 10,000,000,000 down to, at worst,
3,628,800.

> For each tuple returned  by the generator, you can iterate on the set of unique letters and use string functions to substitute the digits, or perhaps do this all at once. You would do this to all the items in what I call the left list as well as all the items on the right list. These would not be "numeric" so using int() on each item  would work EVEN with leading zeroes. Seems safe enough.
>

Use str.replace() and int(), job done.

> Yes, this is brute force. Using range(1,000,000) (no commas in real use)

Use underscores instead:

>>> range(1_000_000)
range(0, 1000000)

> would be a million iterations when there are six unique characters in the puzzle and as many as 10 billion if all ten are in use. If you use nested loops like I showed a while ago (albeit to make arbitrary ones for many sizes could require multiple functions or use of an eval on one built by hand) you can cut down the number of iterations as the nested loops count down with each one doing one less than the one before it. Same goes for the recursive function call method as it passes along what numerals  have already been used. There may already be a permutation function that does this efficiently in C.
>

Yeah, check itertools :)

> The real cost that dominates here is not memory, albeit garbage collection may be busy as it generates lots of temporary small bits of data. It is how the number of iterations grows.

Correct. And that's why a pure brute-force solution needs some
refinement. Algorithmic improvements almost always trump mechanical
improvements.

> I have looked at a somewhat related issue of how you generate all possible SCRABBLE words (or BOGGLE or ...) given specific starting letters.
> One way to make all possible combinations is along the lines above with many needed changes as there can (in theory) be as many as 26 unique letters (in English) and you can have multiple copies of some letters. If you allow other languages, the problem grows to the point where brute force is not realistic.
> And, ideally, you winnow down your choices by checking each word against a large dictionary.

Hmm, IMO that's the wrong way around. Instead, *start* with the
dictionary, and winnow down the possibilities to the words you have. A
decent word list will probably have 100K-1M words in it, which is a
small enough corpus to go through them all.

> Anyone know of one that has a decent selection and can be freely imported? I mean one word per line.
>

What OS are you on? For my usage, I just read from
/usr/share/dict/words and it's usually there. On Debian Linux, that's
provided by the dictionaries-common package, or the wbritish or
wamerican Ditch-side-specific packages. In fact, using that file makes
your code independent of language (although you may need to concern
yourself with different alphabets if you want to support
Russian/Greek, and anything involving "letters" doesn't really work
with Chinese), so I would strongly recommend it.

On Windows, where that path doesn't exist, and there probably aren't
standard dictionaries, you could download one of them from
wordlist.aspell.net or wordlist.sourceforge.net - they're freely
available, but you would have to go fetch them.

> I apologize for the length. The main question was whether eval is particularly expensive.

Well, yes it IS expensive... but the cost of it is less significant
than the algorithm and consequent number of iterations/attempts. Using
eval() on three million possibilities is going to be WAY cheaper than
a more efficient calculation technique used on ten billion.

Write your code with two main priorities:

1) Get your algorithm right
2) Express your algorithm cleanly in code.

Don't worry about performance until you've done the above two steps,
*measured*, and found that it's taking too long.

ChrisA


From 33alperen200233 at gmail.com  Tue Dec 11 13:11:59 2018
From: 33alperen200233 at gmail.com (=?UTF-8?Q?Alperen_Ero=C4=9Flu?=)
Date: Tue, 11 Dec 2018 20:11:59 +0200
Subject: No subject
Message-ID: <CAM18Wf1JhmkuPLL=a4NgNQA69U1FdENVthP7HEWVj+WPKxWLTg@mail.gmail.com>

I subscribed to python list and now I want to state my requeat.I downloaded
Python because I wanted to learn how to code.I also downloaded a text
editor specially designed for coding Python(which iscalled Pycharm)
.both of the softwares and my Windows 10 software was he latest.I got into
Python and it asked me which text file I was going to use to edit files,but
the app(which is Python)couldn't find where the files were located.Hope I
shall get a reply sir

Thank you.


From bgailer at gmail.com  Tue Dec 11 15:43:33 2018
From: bgailer at gmail.com (Bob Gailer)
Date: Tue, 11 Dec 2018 15:43:33 -0500
Subject: No subject
In-Reply-To: <CAM18Wf1JhmkuPLL=a4NgNQA69U1FdENVthP7HEWVj+WPKxWLTg@mail.gmail.com>
References: <CAM18Wf1JhmkuPLL=a4NgNQA69U1FdENVthP7HEWVj+WPKxWLTg@mail.gmail.com>
Message-ID: <CAP1rxO6K7+uh1kgwXfHf-+5-OfuM0XTQrmETO5R-dFgkPg5sHQ@mail.gmail.com>

On Dec 11, 2018 3:05 PM, "Alperen Ero?lu" <33alperen200233 at gmail.com> wrote:
>
> I subscribed to python list and now I want to state my requeat.I
downloaded
> Python because I wanted to learn how to code.I also downloaded a text
> editor specially designed for coding Python(which iscalled Pycharm)
> .both of the softwares and my Windows 10 software was he latest.I got into
> Python

We will need more information in order to help you. What did you do to get
into python?

and it asked me which text file I was going to use to edit files

I'm sorry but that does not make sense to me. Could you show us exactly
what this request looks like?

> the app(which is Python)couldn't find where the files were located.

Again please show us the messages you got. It is best if possible to copy
and paste this information into your reply

Bob Gailer


From avigross at verizon.net  Tue Dec 11 21:06:59 2018
From: avigross at verizon.net (Avi Gross)
Date: Tue, 11 Dec 2018 21:06:59 -0500
Subject: 03 digression by brute force
Message-ID: <007d01d491bf$5dd1ab30$19750190$@verizon.net>

SYNOPSIS: One way to solve math puzzle by brute force. (message sent earlier disappeared)

 

Quick note. Jack started by asking why python does not like decimal numbers with leading zeroes. When asked to explain, he said he was trying to solve word problems using python. Someone mentioned problems like solving SEND + MORE = MONEY and I decided to quickly write a function that would solve anything of that sort that only has addition on either side of the equals.

 

What amused me was that I had 25 solutions to the above when I was told there would be one. Closer examination showed that 24 of those had the ?M? in MONEY set to zero which the logicians claimed was not a sensible solution.

 

This solution is one of a list of 25 dictionaries returned, printed with pprint.print:

 

{' SYNOPSIS': 'PUZZLE: SEND+MORE=MONEY with SOLUTION #1: 7429+0814=08243',

'D': '9',

'E': '4',

'M': '0',

'N': '2',

'O': '8',

'R': '1',

'S': '7',

'Y': '3'}

 

Note M == 0

 

The lone normal answer is:

 

{' SYNOPSIS': 'PUZZLE: SEND+MORE=MONEY with SOLUTION #25: 9567+1085=10652',

'D': '7',

'E': '5',

'M': '1',

'N': '6',

'O': '0',

'R': '8',

'S': '9',

'Y': '2'}

 

Apparently, unless you tell the program to skip any attempts where M is not 1, it finds these. I will spare the rest but include the code if you wish to try it. I called the function this (perhaps odd) way:

 

puzzle = "SeND+ MoRe =    MoNeY???!!!!!!!!!!!!"

solution25 = math_word_puzzle_solver(puzzle)

 

And it returned a list of the 25 dicts.

 

So I tried this puzzle:

 

puzzle = "SeND + MoRe = oNeY"

 

I guessed (wrongly) that by removing the offending M in MONEY, it would find only the other 24 solutions with the M in MORE being 0.

 

Not quite. That got 108 solutions! M was just about anything:

 

[d['M'] for d in solution108 ]

                   

['6', '2', '5', '3', '6', '2', '5', '3', '7', '1', '6', '2', '7', '6', '2', '1', '5', '3', '7', '1', '5', '2', '5', '2', '7', '0', '7', '0', '4', '3', '7', '0', '8', '0', '4', '3', '8', '0', '8', '0', '6', '2', '5', '1', '6', '0', '5', '1', '6', '0', '5', '1', '7', '0', '4', '3', '7', '0', '7', '6', '1', '0', '7', '0', '4', '1', '5', '0', '6', '4', '2', '0', '6', '0', '6', '0', '3', '1', '5', '0', '5', '0', '3', '2', '2', '1', '2', '1', '2', '1', '3', '0', '2', '1', '3', '0', '3', '1', '2', '0', '2', '0', '3', '0', '3', '0', '2', '1']

 

My main point though is that a leading zero can appear including in bank account numbers, social security numbers and so on. But I accept that historically python is as it is. As I mentioned, some functions like int() can deal with them.

 

I attach my code in case anyone is interested in seeing the whole output or trying it on other puzzles. The code uses no exec() or eval() and should be safe ?

 

########################CODE################

 

# Module created to solve a specific kind of math word problem.

 

# FIND all solutions to a puzzle by systematically

# replacing LETTERS with NUMERALS till an equation is True

 

# Accepts string like "SEND + MORE = MONEY"

 

# Measure time elapsed, optional:

def math_word_puzzle_solver(puzzle):

    """USAGE: math_word_puzzle_solver(puzzle)"""

    """Solve a puzzle by brute force"""

    """of the form SEND + MORE = MONEY"""

    """with arbitrary sums on either side of"""

    """the equation."""

      

    # NOTE: comments illustrate what happens to the test case only.

    

    # Assuming the user typed in garbage, will restrict characters and normalize.

    # Remove whitespace and most other things except +/=, make uppercase, etc.

 

    puzzle = puzzle.upper()

    retain = set('ABCDEFGHIJKLMNOPQRSTUVWXYZ=+')

    puzzle = ''.join(filter(retain.__contains__, puzzle))

 

    # puzzle := 'SEND+MORE=money'

 

    # Split into left/right on the equals sign:

 

    left_text, right_text = puzzle.split('=')

 

    # left_text  := 'SEND+MORE'

    # right_text :=  'MONEY' (or 'ONEY')

 

    # Split each side into a list on "+" and in the test case

    # only the left side has more than one:

 

    left_list = left_text.split('+')

    right_list = right_text.split('+')

 

    # At this point we have:

    # left_list: ['SEND', 'MORE']

    # right_list: ['MONEY']

 

    # Combine everything in both lists to make a set with unique letters.

 

    letter_set = { letter

                   for word in (left_list + right_list)

                   for letter in word }

 

    # Letter_set := {'M', 'O', 'R', 'Y', 'N', 'S', 'E', 'D'}

 

    letter_count = len(letter_set)

 

    # letter_count := 8

 

    # Make an iterator to deliver one combination at a time 

    # of the n-tuple of n digits at a time from 0-9 with no replacement

    # and luckily that problem is solved by using the itertools module:

 

    import itertools

 

    # We want all permutations taken letter_count at a time

    # so the iterator will return an 8-tuple in this example looking like:

    #    ('0', '1', '2', '3', '4', '5', '6', '7')

    #    ('0', '1', '2', '3', '4', '5', '6', '8')

    # and so on. If run to completion, 1814400 iterations in this case.

 

    # Next line returns an iterator that will be iterated later.

 

    n_tuples = itertools.permutations( iterable = "0123456789",

                                       r = letter_count)

 

    # Overview to minimize comments below. Loop on all such n-tuples.

    # For each result, use a dictionary to make substitutions

    # from letters to numerals on all the text.

    # Convert number strings  to numbers.

    # Calculate if sum of left numbers == sum of right numbers.

 

    letter_tuple = tuple(letter_set)

 

    # Make a list to hold solutions and count them.

 

    solutions = []

    solution_count = 0

 

    # Now to loop several levels

 

    for n_tuple in n_tuples:

        # Make a translation dictionary for this iteration

        xlate = dict(zip(letter_tuple, n_tuple))

        

        # initialize sums to zero each time.

        sum_left = sum_right = 0

 

        # A tad complex. for each "word" as in "SEND" or "MORE"

        # get the letters in word and replace from the dictionary

        # each letter with a numeral then join the numerals back

        # into a single string and convert that to an integer and

        # add it to the sum. In this case, add something like

        # 9567 + 1085 = 10652

        

        for word in left_list:

            sum_left += int("".join([ xlate[letter]

                                      for letter in word]))

 

        # Ditto for the right list which only happens to have

        # a single occupant but could have more. May have a sum of

        # anything but in one case 10652 is tried.

        

        for word in right_list:

            

            sum_right += int("".join([ xlate[letter] for letter in word]))

 

        # Time to see if the sums match. If so, add the original

        # problem statement and the numeric translation/solution to the dict

        # As well as the solution number.

        

        if sum_left == sum_right:

            solution_count += 1

            # Make a copy of the solution dictionary

            # augmented to replace + and = with themselves.

            temp=xlate.copy()

            temp["="] = "=" 

            temp["+"] = "+"

            solution_xlated = "".join([ temp[letter]

                                       for letter in puzzle])

            annotation = f"PUZZLE: {puzzle} with SOLUTION #{solution_count}: {solution_xlated}"

 

            # Add an annotation to the dictionary

            xlate[" SYNOPSIS"] = annotation

            # Add the solution to the list to be returned later.

            solutions.append(dict(sorted(xlate.items(), key =lambda x: x[0])))

 

    return(solutions)           

    #  END function definition for math_word_puzzle_solver(puzzle)

 

 

if __name__ == "__main__":

    # This is being run as a program so,

    # calling the function with the default puzzle

    

    import time, pprint

    

    start_time = time.time()

 

    print("=============TWENTY FIVE solutions expected")

    print("=============TWENTY FOUR have M==0")

 

    puzzle = "SeND+ MoRe =    MoNeY???!!!!!!!!!!!!"

        

    solution25 = math_word_puzzle_solver(puzzle)

 

    print("======\nELAPSED TIME: ", time.time() - start_time)

    pprint.pprint( solution25 )

    

    puzzle = "SeND + MoRe = oNeY"

 

    solution108 = math_word_puzzle_solver(puzzle)

 

    print("======\nELAPSED TIME: ", time.time() - start_time)

    

    pprint.pprint( solution108 )

 

#########NO MORE CODE#################

 

    

        

 

 

 

 

 



From nad at python.org  Tue Dec 11 22:14:04 2018
From: nad at python.org (Ned Deily)
Date: Tue, 11 Dec 2018 22:14:04 -0500
Subject: [RELEASE] Python 3.7.2rc1 and 3.6.8rc1 now available for testing  
Message-ID: <091E715F-EC84-4796-B193-D96414F4D62E@python.org>

https://blog.python.org/2018/12/python-372rc1-and-368rc1-now-available.html


Python 3.7.2rc1 and 3.6.8rc1 are now available. 3.7.2rc1 is the release
preview of the next maintenance release of Python 3.7, the latest
feature release of Python. 3.6.8rc1 is the release preview of the next
and last maintenance release of Python 3.6, the previous feature
release of Python. Assuming no critical problems are found prior to
2018-12-20, no code changes are planned between these release
candidates and the final releases. These release candidates are
intended to give you the opportunity to test the new security and bug
fixes in 3.7.2 and 3.6.8. We strongly encourage you to test your
projects and report issues found to bugs.python.org as soon as
possible. Please keep in mind that these are preview releases and,
thus, their use is not recommended for production environments.

You can find these releases and more information here:
    https://www.python.org/downloads/release/python-372rc1/
    https://www.python.org/downloads/release/python-368rc1/


--
  Ned Deily
  nad at python.org -- []



From python at mrabarnett.plus.com  Tue Dec 11 22:54:14 2018
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 12 Dec 2018 03:54:14 +0000
Subject: 03 digression by brute force
In-Reply-To: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
Message-ID: <8b852cee-7b6d-d68c-19cd-d8e4e4eba6ed@mrabarnett.plus.com>

On 2018-12-12 02:06, Avi Gross wrote:
[snip]
> My main point though is that a leading zero can appear including in bank account numbers, social security numbers and so on. But I accept that historically python is as it is. As I mentioned, some functions like int() can deal with them.
> 
Bank account numbers, etc, are not really numbers, but are, instead, 
identifiers. You wouldn't do arithmetic with such a number, nor should 
you store one in a integer field in a database. Dropping a leading digit 
would be like dropping a leading letter of a name.


From pfeiffer at cs.nmsu.edu  Wed Dec 12 00:03:42 2018
From: pfeiffer at cs.nmsu.edu (Joe Pfeiffer)
Date: Tue, 11 Dec 2018 22:03:42 -0700
Subject: 03 digression by brute force
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
Message-ID: <1befancp5d.fsf@pfeifferfamily.net>

"Avi Gross" <avigross at verizon.net> writes:

> SYNOPSIS: One way to solve math puzzle by brute force. (message sent earlier disappeared)
>
>  
>
> Quick note. Jack started by asking why python does not like decimal
> numbers with leading zeroes. When asked to explain, he said he was
> trying to solve word problems using python. Someone mentioned problems
> like solving SEND + MORE = MONEY and I decided to quickly write a
> function that would solve anything of that sort that only has addition
> on either side of the equals.
>
>  
>
> What amused me was that I had 25 solutions to the above when I was
> told there would be one. Closer examination showed that 24 of those
> had the ?M? in MONEY set to zero which the logicians claimed was not a
> sensible solution.

What amuses me is the solution to the problem that started the whole
thread had at least one number with a leading 0.


From fabian.becker87 at gmail.com  Wed Dec 12 00:30:10 2018
From: fabian.becker87 at gmail.com (fabian.becker87 at gmail.com)
Date: Tue, 11 Dec 2018 21:30:10 -0800 (PST)
Subject: How to test for type or instance of dict_values?
In-Reply-To: <mailman.9.1479402549.20410.python-list@python.org>
References: <o0k7j9$29r$1@blaine.gmane.org> <o0k8c1$g3o$1@blaine.gmane.org>
 <o0kgh9$pd6$1@blaine.gmane.org> <o0ko78$3um$1@blaine.gmane.org>
 <mailman.9.1479402549.20410.python-list@python.org>
Message-ID: <91d67b3c-4909-46cd-8e37-4210afda6830@googlegroups.com>

Very late to the party, but I just encountered a very similar problem and found a solution:

```
import collections

obj = {"foo": "bar"}
isinstance(obj.values(), collections.abc.ValuesView) # => True
```

Hope that helps someone out there :)

On Thursday, November 17, 2016 at 9:09:23 AM UTC-8, Terry Reedy wrote:
> On 11/17/2016 9:57 AM, Thorsten Kampe wrote:
> 
> > The code in question is part of an attempt to get the dimensions of
> > multi-dimensional lists, the `isinstance` is there in order to
> > exclude strings.
> 
> You can do the exclusion directly.
> 
> >
> > """
> > def dim(seq):
> >     dimension = []
> >     while isinstance(seq, (list, tuple)):
> 
>      while not isinstance(seq, str) # or (str, bytes, ...)
> 
> >         dimension.append(len(seq))
> >         try:
> >             seq = seq[0]
> >         except IndexError:  # sequence is empty
> >             break
> >     return dimension
> > """
> >
> > Thorsten
> >
> 
> 
> -- 
> Terry Jan Reedy


From avigross at verizon.net  Wed Dec 12 00:46:03 2018
From: avigross at verizon.net (Avi Gross)
Date: Wed, 12 Dec 2018 00:46:03 -0500
Subject: zeroed out
Message-ID: <000b01d491dd$f84023e0$e8c06ba0$@verizon.net>

Joe,

All numbers start with zero, and even an infinite number of them! LOL!

All kidding aside, I note that some data that is stored in a fixed width has zeroes padding it all the way to the right. If you store the number 3 in binary as a byte, it tends to look like 00000011. Depending on how integers are stored, something similar can happen for longer stretches. But with decimals, we tend to not bother showing the empty areas that represent 0 times ever higher powers of 10.

But on the right, zeroes are required as otherwise you have no idea what the other numerals mean. 5900 is not 59.  Yes, you can write 59e2 or 5.9e3. Yet on the other side of the decimal point, it is reversed. You need 0's on the left but on the right only if you want to show significance.

But there are limits. Most programs do not support indefinite precision arithmetic. Too many digits including zeroies tend to be ignored.

Python integers though, see boundless:

>>> 2**80
1208925819614629174706176
>>> _**10
6668014432879854274079851790721257797144758322315908160396257811764037237817632071521432200871554290742929910593433240445888801654119365080363356052330830046095157579514014558463078285911814024728965016135886601981690748037476461291163877376
>>> _**10
173766203193809456599982445949435627061939786100117250547173286503262376022458008465094333630120854338003194362163007597987225472483598640843335685441710193966274131338557192586399006789292714554767500194796127964596906605976605873665859580600161998556511368530960400907199253450604168622770350228527124626728538626805418833470107651091641919900725415994689920112219170907023561354484047025713734651608777544579846111001059482132180956689444108315785401642188044178788629853592228467331730519810763559577944882016286493908631503101121166109571682295769470379514531105239965209245314082665518579335511291525230373316486697786532335206274149240813489201828773854353041855598709390675430960381072270432383913542702130202430186637321862331068861776780211082856984506050024895394320139435868484643843368002496089956046419964019877586845530207748994394501505588146979082629871366088121763790555364513243984244004147636040219136443410377798011608722717131323621700159335786445601947601694025107888293017058178562647175461026384343438874861406516767158373279032321096262126551620255666605185789463207944391905756886829667520553014724372245300878786091700563444079107099009003380230356461989260377273986023281444076082783406824471703499844642915587790146384758051663547775336021829171033411043796977042190519657861762804226147480755555085278062866268677842432851421790544407006581148631979148571299417963950579210719961422405768071335213324842709316205032078384168750091017964584060285240107161561019930505687950233196051962261970932008838279760834318101044311710769457048672103958655016388894770892065267451228938951370237422841366052736174160431593023473217066764172949768821843606479073866252864377064398085101223216558344281956767163876579889759124956035672317578122141070933058555310274598884089982879647974020264495921703064439532898207943134374576254840272047075633856749514044298135927611328433323640657533550512376900773273703275329924651465759145114579174356770593439987135755889403613364529029604049868233807295134382284730745937309910703657676103447124097631074153287120040247837143656624045055614076111832245239612708339272798262887437416818440064925049838443370805645609424314780108030016683461562597569371539974003402697903023830108053034645133078208043917492087248958344081026378788915528519967248989338592027124423914083391771884524464968645052058218151010508471258285907685355807229880747677634789376


-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Joe Pfeiffer
Sent: Wednesday, December 12, 2018 12:04 AM
To: python-list at python.org
Subject: Re: 03 digression by brute force

"Avi Gross" <avigross at verizon.net> writes:

> SYNOPSIS: One way to solve math puzzle by brute force. (message sent 
> earlier disappeared)
>
>  
>
> Quick note. Jack started by asking why python does not like decimal 
> numbers with leading zeroes. When asked to explain, he said he was 
> trying to solve word problems using python. Someone mentioned problems 
> like solving SEND + MORE = MONEY and I decided to quickly write a 
> function that would solve anything of that sort that only has addition 
> on either side of the equals.
>
>  
>
> What amused me was that I had 25 solutions to the above when I was 
> told there would be one. Closer examination showed that 24 of those 
> had the ?M? in MONEY set to zero which the logicians claimed was not a 
> sensible solution.

What amuses me is the solution to the problem that started the whole thread had at least one number with a leading 0.
--
https://mail.python.org/mailman/listinfo/python-list



From songbird at anthive.com  Wed Dec 12 12:04:46 2018
From: songbird at anthive.com (songbird)
Date: Wed, 12 Dec 2018 12:04:46 -0500
Subject: setup.py and licensing questions
Message-ID: <emr7ef-pq4.ln1@anthive.com>

hi,

  i'm currently working through all the packaging and 
licensing stuff for my project and want to put the 
license in the setup.py file, but there may actually 
be more than one license (GPL ones for the artwork 
i've borrowed from another project and whatever license 
i choose to put on my own code and the artwork i've done).

  can i put multiple License lines in setup.py 
classifiers like: ?

          "License :: OSI Approved :: ???",
          "License :: OSI Approved :: ???",


  thanks,


  songbird


From afriyie.abraham at gmail.com  Wed Dec 12 13:41:47 2018
From: afriyie.abraham at gmail.com (Afriyie Abraham Kwabena)
Date: Wed, 12 Dec 2018 20:41:47 +0200
Subject: Help - Python REST API 
Message-ID: <DD0FE710-1DBF-4D63-8023-9B335DFE0059@gmail.com>

Hi,

Am trying to create a basic authorization server using client credentials flow and client so that the
client can send it credentials using REST methods (POST) to the authorization server. Authrization server then 
respond with access token. After the access, client can POST, GET, etc to the server for resources.

Can anyone help me or give guidelines how I can achieve this using python?
Am new in python (average) and need some guidelines or some links.

BR
Abraham

From python at mrabarnett.plus.com  Wed Dec 12 14:35:26 2018
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 12 Dec 2018 19:35:26 +0000
Subject: zeroed out
In-Reply-To: <6uk21e9gkqe0ronmopakq1jjrnsi8qvvs5@4ax.com>
References: <000b01d491dd$f84023e0$e8c06ba0$@verizon.net>
 <6uk21e9gkqe0ronmopakq1jjrnsi8qvvs5@4ax.com>
Message-ID: <33a3ef36-e94d-c4ae-b91a-6e7228b2a121@mrabarnett.plus.com>

On 2018-12-12 18:59, Dennis Lee Bieber wrote:
> On Wed, 12 Dec 2018 00:46:03 -0500, "Avi Gross" <avigross at verizon.net>
> declaimed the following:
> 
>>
>>All kidding aside, I note that some data that is stored in a fixed width has zeroes padding it all the way to the right. If you store the number 3 in binary as a byte, it tends to look like 00000011. Depending on how integers are stored, something similar can happen for longer stretches. But with decimals, we tend to not bother showing the empty areas that represent 0 times ever higher powers of 10.
>>
> 
> 	Original COBOL implementation performs arithmetic on BCD (more likely,
> packed BCD). With two decimal digits per byte (packed BCD), a common
> 32-digit number requires 16 bytes, or 4 longwords (given the size of
> processors in 1960). Since arithmetic is basically an array operation in
> BCD (process least significant digit, process next digit..., etc.) numbers
> required storage of the filler zeros.
> 
> 	Furthermore, of more significance, formatting of a BCD number for
> output merely required taking each BCD digit (unpacked) or nybble (packed)
> and ADDING the value of (ASCII) "0". Conversely, input of (ASCII) fixed
> width/filled numbers simply requires subtracting "0" from each digit to get
> the BCD value for it (followed by packing the right-side nybbles to get
> packed BCD)
> 
> 	The Intel 8080 includes a DAA (decimal adjust accumulator) instruction
> to "correct" an 8-bit value back to packed BCD. Adding two 2-digit BCD
> values using normal binary ADD/ADC would result in an incorrect binary
> value (the source wasn't binary, but the result isn't BCD), DAA would
> adjust that back to BCD, using a "half carry" flag to handle the internal
> digit split.
> 
Later processors have a DAS instruction, which is used after BCD 
subtraction.

The humble 6502 doesn't have DAA/DAS, but instead has a decimal mode flag.

There's a clever way to perform BCD addition on long BCD numbers 
(32-bit, 64-bit, whatever) without having to do it 1 digit (or 1 byte) 
at a time.


From ben+python at benfinney.id.au  Wed Dec 12 20:27:15 2018
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 13 Dec 2018 12:27:15 +1100
Subject: setup.py and licensing questions
References: <emr7ef-pq4.ln1@anthive.com>
Message-ID: <861s6m1aj0.fsf@benfinney.id.au>

songbird <songbird at anthive.com> writes:

>   can i put multiple License lines in setup.py 
> classifiers like: ?
>
>           "License :: OSI Approved :: ???",
>           "License :: OSI Approved :: ???",

Yes.

The semantics of that are not formalised, to my knowledge. You would be
signaling that the specified licenses are involved somehow, but not how
they combine for this specific work.

For this reason (and others) you should always put an explicit,
unambiguous *grant of license* in the work, preferably in the README
document or somewhere prominent like that.

The grant of license is a statement, preferably brief, that says exactly
what the work is, who holds copyright, who receives a grant of license,
and what that grants the recipient to do, under what explicit conditions.

For example:

    Copyright ? 2018 Ben Finney <ben+python at benfinney.id.au>
    Purple Drachma is free software: you are free to copy, modify,
    and/or distribute this work under the conditions of the GNU Affero
    General Public License, version 3 or later as published by the Free
    Software Foundation. See the file LICENSE.AGPL-3.0 for details.

-- 
 \     ?You don't need a book of any description to help you have some |
  `\    kind of moral awareness.? ?Dr. Francesca Stavrakoloulou, bible |
_o__)                                              scholar, 2011-05-08 |
Ben Finney



From ben+python at benfinney.id.au  Wed Dec 12 20:30:29 2018
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 13 Dec 2018 12:30:29 +1100
Subject: Help - Python REST API
References: <DD0FE710-1DBF-4D63-8023-9B335DFE0059@gmail.com>
Message-ID: <86wooez00a.fsf@benfinney.id.au>

Afriyie Abraham Kwabena <afriyie.abraham at gmail.com> writes:

> Am trying to create a basic authorization server using client
> credentials flow and client so that the client can send it credentials
> using REST methods (POST) to the authorization server.

At minimum, this implies that you are implementing a web server (either
from scratch, or by plugging your code into some existing web server) to
listen for and respond to the HTTP requests.

Have you already chosen which web server will be receiving requests?

> Can anyone help me or give guidelines how I can achieve this using
> python?

There are existing frameworks (Flask, Django) that make it relatively
straightforward to implement a REST API ? that is, relative to writing
it from scratch!

Even using such a framework, you would still be accepting the burden of
implementing and deploying a full web service.

-- 
 \     ?Computers are useless. They can only give you answers.? ?Pablo |
  `\                                                           Picasso |
_o__)                                                                  |
Ben Finney



From songbird at anthive.com  Wed Dec 12 22:31:11 2018
From: songbird at anthive.com (songbird)
Date: Wed, 12 Dec 2018 22:31:11 -0500
Subject: setup.py and licensing questions
References: <emr7ef-pq4.ln1@anthive.com> <861s6m1aj0.fsf@benfinney.id.au>
 <mailman.39.1544664445.2771.python-list@python.org>
Message-ID: <vc09ef-p9b.ln1@anthive.com>

Ben Finney wrote:
> songbird writes:
>
>>   can i put multiple License lines in setup.py 
>> classifiers like: ?
>>
>>           "License :: OSI Approved :: ???",
>>           "License :: OSI Approved :: ???",
>
> Yes.
>
> The semantics of that are not formalised, to my knowledge. You would be
> signaling that the specified licenses are involved somehow, but not how
> they combine for this specific work.

  ok, thanks, that's good to know.

  i will have a list of each file i've used from
elsewhere, where it came from and as best as i can 
figure it out the licenses/copyright holders and/or
permissions.  i'm not using anyone else's code but
i am using some artwork and a man page that i've
adapted.


> For this reason (and others) you should always put an explicit,
> unambiguous *grant of license* in the work, preferably in the README
> document or somewhere prominent like that.
>
> The grant of license is a statement, preferably brief, that says exactly
> what the work is, who holds copyright, who receives a grant of license,
> and what that grants the recipient to do, under what explicit conditions.
>
> For example:
>
>     Copyright ? 2018 Ben Finney <ben+python at benfinney.id.au>
>     Purple Drachma is free software: you are free to copy, modify,
>     and/or distribute this work under the conditions of the GNU Affero
>     General Public License, version 3 or later as published by the Free
>     Software Foundation. See the file LICENSE.AGPL-3.0 for details.

  yes, i'll be putting a license on the code and any
of the artwork i've made.

  thank you,


  songbird


From venkatesh.jonnakuti at gmail.com  Wed Dec 12 23:25:21 2018
From: venkatesh.jonnakuti at gmail.com (VENKEY ROKZS)
Date: Wed, 12 Dec 2018 20:25:21 -0800 (PST)
Subject: When to Use Django & Flask frame work
Message-ID: <2f0cb0e0-9996-45ba-8071-194315da4b38@googlegroups.com>

Hi Viewers,

 I am new to java I want to about when to Use Django & Flask.. I have started working on it.Anybody can you suggest the difference...will be Appreciated.


From ben+python at benfinney.id.au  Wed Dec 12 23:44:47 2018
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 13 Dec 2018 15:44:47 +1100
Subject: When to Use Django & Flask frame work
References: <2f0cb0e0-9996-45ba-8071-194315da4b38@googlegroups.com>
Message-ID: <86sgz2yr0g.fsf@benfinney.id.au>

VENKEY ROKZS <venkatesh.jonnakuti at gmail.com> writes:

> Hi Viewers,

Welcome to the Python discussion forum.

>  I am new to java I want to about when to Use Django & Flask..

Typically we use those frameworks more with Python, not Java :-)

> I have started working on it.Anybody can you suggest the difference...

The biggest difference I would point out: Flask is small, Django is big.

More accurately: Both are full web development frameworks, but Flask is
focussed more to allow you to assemble many of the pieces yourself,
which Django makes many of the decisions for you by default.

Even more accurately: Both Flask and Django are reasonably small and
fast, and both allow you to customise the pieces quite well. The
difference I alluded to is rather one of emphasis: Flask expects that
you will want to quite soon replace many of the standard pieces, while
Django expects that you can go the whole application lifecycle without
needing to change any of the default pieces.

I hope that helps! If you have the time, feel free to evaluate each of
them and choose whichever one suits your need. If you don't have time, I
would recommend starting with Django because you will be able to get a
long way without needing to diverge from the default.

-- 
 \          ?If I ever get real rich, I hope I'm not real mean to poor |
  `\                              people, like I am now.? ?Jack Handey |
_o__)                                                                  |
Ben Finney



From greg.ewing at canterbury.ac.nz  Thu Dec 13 01:14:26 2018
From: greg.ewing at canterbury.ac.nz (Gregory Ewing)
Date: Thu, 13 Dec 2018 19:14:26 +1300
Subject: zeroed out
In-Reply-To: <mailman.38.1544643515.2771.python-list@python.org>
References: <000b01d491dd$f84023e0$e8c06ba0$@verizon.net>
 <6uk21e9gkqe0ronmopakq1jjrnsi8qvvs5@4ax.com>
 <33a3ef36-e94d-c4ae-b91a-6e7228b2a121@mrabarnett.plus.com>
 <mailman.38.1544643515.2771.python-list@python.org>
Message-ID: <g7ebm4Fm423U1@mid.individual.net>

MRAB wrote:
> Later processors have a DAS instruction, which is used after BCD 
> subtraction.
> 
> The humble 6502 doesn't have DAA/DAS, but instead has a decimal mode flag.

The 68000 also had a Decimal Add instruction, but disappointingly
it only worked a byte at a time. I guess running COBOL at high
speed wasn't a priority by then.

But for sheer unadulterated decimalness, you can't beat the
custom CPU that HP used in their handheld calculators in the
70s and 80s. Early versions of it did *only* decimal arithmetic
(on 14-digit registers and memory locations). An option for
binary wasn't added until the HP-41.

-- 
Greg


From dieter at handshake.de  Thu Dec 13 03:33:03 2018
From: dieter at handshake.de (dieter)
Date: Thu, 13 Dec 2018 09:33:03 +0100
Subject: Help - Python REST API
References: <DD0FE710-1DBF-4D63-8023-9B335DFE0059@gmail.com>
Message-ID: <87wood6d34.fsf@handshake.de>

Afriyie Abraham Kwabena <afriyie.abraham at gmail.com> writes:
> Am trying to create a basic authorization server using client credentials flow and client so that the
> client can send it credentials using REST methods (POST) to the authorization server. Authrization server then 
> respond with access token. After the access, client can POST, GET, etc to the server for resources.
>
> Can anyone help me or give guidelines how I can achieve this using python?
> Am new in python (average) and need some guidelines or some links.

As Ben already has pointed out, you have various options for the
choice of the web application framework. I add "Zope[2]" to these options,
potentially together with "Products.PluggableAuthService" and
"dm.zope.rpc".

"Zope" is a so called web application framework which comes with everything
typically required for web applications: url traversal, authentication,
publishing, persistent storage in an object oriented database (the "ZODB",
"Zope Object DataBase"), transaction system, permissions and roles,
user management, ...
The real application comes into existence either by putting objects
into the ZODB or by registering "view"s with the framework.

The framework can be easily extended.
"Products.PluggableAuthService" is such an extension to get
an extremely flexible subframework for special authentication
and authorization needs.
"dm.zope.rpc" is another extension which facilitates the
implementation of "rpc" (= "Remote Procedure Call") services with
Zope based on different protocols.





From ammon.hepworth at gmail.com  Thu Dec 13 11:00:50 2018
From: ammon.hepworth at gmail.com (ammon.hepworth at gmail.com)
Date: Thu, 13 Dec 2018 08:00:50 -0800 (PST)
Subject: python CAD libraries?
In-Reply-To: <e149e0ec-39db-4c90-af60-2353ca923c2f@googlegroups.com>
References: <e149e0ec-39db-4c90-af60-2353ca923c2f@googlegroups.com>
Message-ID: <90606416-ea35-42a8-a5dc-1a7b56220547@googlegroups.com>

I know this is an old post, but I wanted to recommend another Python CAD library
called Pascale: https://aerion-tech.com/pascale

It creates commercial grade CAD geometry through an intuitive Python API. Although it is a paid software, but there is a free version available to use on the Cloud.


From garyfallidis at gmail.com  Thu Dec 13 11:38:35 2018
From: garyfallidis at gmail.com (Eleftherios Garyfallidis)
Date: Thu, 13 Dec 2018 11:38:35 -0500
Subject: ANN: DIPY 0.15.0 release
Message-ID: <CAN6JmWPWGakqkRLud_OgnDZe85vKOLma8Spo4ZazxtHwVt9twg@mail.gmail.com>

We are excited to announce a new release of Diffusion Imaging in Python
(DIPY).

DIPY 0.15 (Wednesday, 12 December 2018)

This release received contributions from 30 developers (the full release
notes are at: http://dipy.org/release0.15.html
<http://nipy.org/dipy/release0.12.html>)

Highlights of this release include:


   -

   Updated RecoBundles for automatic anatomical bundle segmentation.
   -

   New Reconstruction Model: qtau-dMRI signal representation.
   -

   New command line interfaces (e.g., dipy_slr).
   -

   New continuous integration with AppVeyor CI.
   -

   Nibabel Streamlines API now used almost everywhere for better memory
   management.
   -

   Compatibility with Python 3.7.
   -

   Many tutorials added or updated (5 New).
   -

   Large overall documentation update.
   -

   Moved visualization module to a new library: FURY.
   -

   Closed 287 issues and merged 93 pull requests.


Want to learn more?

The DIPY project is hosting an educational workshop March 11-15 at Indiana
University, USA. The event is sponsored by the National Institute of Health
(NIH). See topics and detailed information here:

https://workshop.dipy.org

The workshop will include tutorials on tracking, microstructure, and
statistical analysis. The workshop will also include a data accelerator
where you will be able to analyze your own data together with the DIPY
<http://dipy.org/> developers. More than 15 DIPY developers will be
available this year on site to help. All tutorials will run on 3 levels:
novice, intermediate and advanced. Register here <https://workshop.dipy.org>
.


To upgrade or install <http://dipy.org/release0.10.html> DIPY

Run the following command in your terminal:
<http://dipy.org/release0.10.html>

pip install --upgrade dipy

or

conda install -c conda-forge dipy

This version of DIPY depends on nibabel (2.3.0+).

Questions or suggestions?

For any questions go to http://dipy.org, or send an e-mail to
dipy at python.org <neuroimaging at python.org>

We also have an instant messaging service and chat room available at
https://gitter.im/nipy/dipy

On behalf of the DIPY developers,

Eleftherios Garyfallidis, Ariel Rokem, Serge Koudoro

http://dipy.org/developers.html


From afriyie.abraham at gmail.com  Thu Dec 13 13:00:37 2018
From: afriyie.abraham at gmail.com (Afriyie Abraham Kwabena)
Date: Thu, 13 Dec 2018 20:00:37 +0200
Subject: Python-list Digest, Vol 183, Issue 13
In-Reply-To: <mailman.26.1544720403.26714.python-list@python.org>
References: <mailman.26.1544720403.26714.python-list@python.org>
Message-ID: <A65F72C1-185D-4F64-8CC1-CC65551C4DF6@gmail.com>

Hi,

Thanks for the reply maybe I did not explain myself very well but is it possible to apply this scenario to a database server like MySQL where I can write some python code to post, get, delete, etc to the database? However the client (python code I have to write ) have to be authenticated and then after that can have access to the data in the server. If this is possible what should be my starting point and which python frameworks or libraries do i have to use to achieve this. 

Thanks in advance. 

BR
Abraham

Sent from my iPhone

> On 13 Dec 2018, at 19.00, python-list-request at python.org wrote:
> 
> Send Python-list mailing list submissions to
>    python-list at python.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>    https://mail.python.org/mailman/listinfo/python-list
> or, via email, send a message with subject or body 'help' to
>    python-list-request at python.org
> 
> You can reach the person managing the list at
>    python-list-owner at python.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-list digest..."
> Today's Topics:
> 
>   1. setup.py and licensing questions (songbird)
>   2. Help - Python REST API  (Afriyie Abraham Kwabena)
>   3. Re: zeroed out (Dennis Lee Bieber)
>   4. Re: zeroed out (MRAB)
>   5. Re: setup.py and licensing questions (Ben Finney)
>   6. Re: Help - Python REST API (Ben Finney)
>   7. Re: setup.py and licensing questions (songbird)
>   8. When to Use Django & Flask frame work (VENKEY ROKZS)
>   9. Re: When to Use Django & Flask frame work (Ben Finney)
>  10. Re: zeroed out (Gregory Ewing)
>  11. Re: Help - Python REST API (dieter)
>  12. Re: python CAD libraries? (ammon.hepworth at gmail.com)
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> <mime-attachment>
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From jfong at ms4.hinet.net  Thu Dec 13 21:24:47 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Thu, 13 Dec 2018 18:24:47 -0800 (PST)
Subject: 03 digression by brute force
In-Reply-To: <mailman.32.1544580423.2771.python-list@python.org>
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
Message-ID: <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>

Just for fun:-) On my ooold PC, it takes 0.047 seconds to run the following algorithm on the problem 'SNED + MORE == MONEY".

-----------------------------
import time
import itertools

#S, E, N, D, M, O, R, Y
n = 0
digits = {x for x in range(10)}

def tenThousand(u, Cin):  # Cin == M
    global n
    if Cin == M:
        print(S, E, N, D, '+', M, O, R, E, '==', M, O, N, E, Y)
        n += 1

def thousand(u, Cin):  # Cin + S + M == 10 * Cout + O
    global S, M, n
    rest = digits - set(u)
    for g in itertools.permutations(rest, 2):
        for Cout in range(2):
            if Cin + g[0] + g[1] == 10 * Cout + O:
                S = g[0];  M = g[1]
                tenThousand(u + g, Cout)
            
def hundred(u, Cin):  # Cin + E + O == 10 * Cout + N
    global O, n
    rest = digits - set(u)
    for g in itertools.permutations(rest, 1):
        for Cout in range(2):
            if Cin + E + g[0] == 10 * Cout + N:
                O = g[0]
                thousand(u + g, Cout)

def ten(u, Cin):  # Cin + N + R == 10 * Cout + E
    global N, R, n
    rest = digits - set(u)
    for g in itertools.permutations(rest, 2):
        for Cout in range(2):
            if Cin + g[0] + g[1] == 10 * Cout + E:
                N = g[0];  R = g[1]
                hundred(u + g, Cout)
            
def unit():  # D + E == 10 * Cout + Y
    global D, E, Y, n
    n = 0
    for g in itertools.permutations(range(10), 3):  # g is a tuple
        for Cout in range(2):  # add two items so Cout is 0 or 1
            if g[0] + g[1] == 10 * Cout + g[2]:
                D = g[0];  E = g[1];  Y = g[2]
                ten(g, Cout)
    print(n)
        
if __name__ == '__main__':
    start = time.time()
    unit()
    print(time.time() - start)
-------------------------

--Jach


From dieter at handshake.de  Fri Dec 14 02:42:48 2018
From: dieter at handshake.de (dieter)
Date: Fri, 14 Dec 2018 08:42:48 +0100
Subject: Help - Python REST API (war: Python-list Digest, Vol 183,
 Issue 13)
References: <mailman.26.1544720403.26714.python-list@python.org>
 <A65F72C1-185D-4F64-8CC1-CC65551C4DF6@gmail.com>
Message-ID: <87woocd05j.fsf_-_@handshake.de>

Afriyie Abraham Kwabena <afriyie.abraham at gmail.com> writes:
> ...
> Thanks for the reply maybe I did not explain myself very well but is it possible to apply this scenario to a database server like MySQL where I can write some python code to post, get, delete, etc to the database? However the client (python code I have to write ) have to be authenticated and then after that can have access to the data in the server. If this is possible what should be my starting point and which python frameworks or libraries do i have to use to achieve this. 

I have put "Zope" into the ring (for the server side). If your actual
data lives in a relational database, then "Zope" might not be ideal:
while "dm.zope.reseller" demonstrates that "Zope" can be used as an interface
to entities in a relational database, this is not directly supported
and you need some deeper knowledge to get it done.

The particularity of REST is its very limited support for verbs: it knows
only the standard HTTP commands ("GET", "HEAD", "POST", "PUT", "DELETE")
as verbs. You must introduce artificial objects to support a richer
verb set. E.g. to express the verbs "search_users", "add_user",
you would introduce an object "users" and map "search_users --> users/GET",
"add_user --> users/POST"; for "update_user", you would have user objects
representing an individual user and map "update_user --> <user>/PUT".
If you look at this in the relational world, then the object "users"
corresponds to a table, while a user object corresponds to an entry
in this table, i.e. those objects live on different representational
levels (table, table entry) in the relational database.

Out of the box, Zope only has limited support for this kind
of modelization: its "ZSQL Method"s can be used to represent a table;
their "direct traversal" can be used to represent a table entry.
But, you must do special things to get the REST expected semantics
of "POST", "PUT" and "DELETE".


The client side should be easy -- at least, if the number of objects
you must support is small. You can then use the module "urllib2/http.client"
likely together with the module "json".





From __peter__ at web.de  Fri Dec 14 03:21:23 2018
From: __peter__ at web.de (Peter Otten)
Date: Fri, 14 Dec 2018 09:21:23 +0100
Subject: Smarter algo, was Re: 03 digression by brute force
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
 <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>
Message-ID: <puvp23$v6r$1@blaine.gmane.org>

jfong at ms4.hinet.net wrote:

> Just for fun:-) On my ooold PC, it takes 0.047 seconds to run the
> following algorithm on the problem 'SNED + MORE == MONEY".

> def tenThousand(u, Cin):  # Cin == M
>     global n
>     if Cin == M:
>         print(S, E, N, D, '+', M, O, R, E, '==', M, O, N, E, Y)
>         n += 1

And it probably took under two minutes with the brute force approach.
So if you were only interested in the result the code below were efficient 
only if it took at most two more minutes to write it;)

But seriously, the next step is to generalize this to work with arbitrary 
additions...
More fun!




From avigross at verizon.net  Fri Dec 14 19:13:18 2018
From: avigross at verizon.net (Avi Gross)
Date: Fri, 14 Dec 2018 19:13:18 -0500
Subject: Smarter algo, was Re: 03 digression by brute force
In-Reply-To: <puvp23$v6r$1@blaine.gmane.org>
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
 <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>
 <puvp23$v6r$1@blaine.gmane.org>
Message-ID: <012201d4940a$fb490f00$f1db2d00$@verizon.net>

REAL SUBJECT: Analysis of alternate algorithms.

Peter & Jach and anyone interested,

As Peter said in his altered subject line, Jack changed directions from tweaking an algorithm to trying something quite different.

Reminder of the problem. 

Horizontal View:
SEND + MORE = MONEY

Vertical View:
  SEND
+MORE
...........
MONEY

Hard to be precise as I am sticking to plain text in my message. The three words above are meant to be right adjusted.

When solving it horizontally, Jach and I used variants of a brute force approach. Substitute all possible combinations. He did it in-line and used eval. I did it by making lists of items on both sides and summing the int() of each component and comparing. We tried both our algorithms and his was slower and he profiled that the cause was that eval() was much more expensive as were his use of regular expression functions. For comparison, mine used int() and string manipulation functions and sets and dictionaries.

But the real puzzle is meant to be solved in a more vertical way by humans using logic. I won't do that here but note we have 4 equations going down but 8 unknowns. And the equations are not unique.

The rightmost column (I will call it the first as our arithmetic proceeds from right to left) is meant to represent ONES and provides the equation:

(D+E== Y) or (D+E == Y + 10)

Worse, for the next column, you either get a "1" carried from the previous addition or not and either pass a "1" along to the next column or not. 4 Possibilities.

(N+R==E) or (N+R+1==E) or (N+R==E+10) or (N+R+1==E+10)

Getting a headache yet?

For a human, they need a way to come up with additional info in terms of constraints.

There is a guiding inequality that looks like this:

S is not the same as any of the others. Anytime you solve for another, the list of possible values for S shrinks.
Ditto for each other variable.
Or, since anything plus 0 is itself, then D and E adding up to Y (with no possible carry) cannot be 0.

But getting a computer to do this might be a bit harder than blunt-force searches. So let's see why Jach's new algorithm was faster.

The code I am analyzing can be viewed in the archives and will not be entered again:

https://mail.python.org/pipermail/python-list/2018-December/738454.html

So what did Jach try in his newer version? It is what I call a vertical approach but one a computer can do easier than a human can or would. I see it is a very specific algorithm that hard codes in these variables as global entities that are altered by a set of nested functions. S, E, N, D, M, O, R, Y. There are approaches that might be better such as passing a dictionary partially filled out from one function to the next as the only one that prints the solution is the final function call.

So his is not a general solution.

What interests me as a probable reason this is faster is the number of situations it tries. The earlier versions asked itertools.permutations() to provide all unique combinations of ten tokens in eight positions. So there were 10 choices for the first and 9 for the second and so on adding up to 10!/2! or 1,814,400  different combinations tried. That approaches 2 million.

Jack broke the problem down into evaluating the units column with a loop like this:

itertools.permutations(range(10), 3)

That is 720 possibilities. He then doubled that to 1,440 to consider a carry. Only if the selected values for the three variables in contention (plus a carry) does he go on to call to evaluate the tens column.

It then shrinks a bit more as he no longer gets the permutations of all 10 digits. He subtracts the three values that might work for the units, so he is asking for permutations of 7 digits, two at a time. That is 42, doubled again to 84 for carry purposes. And this function is not called 1,440 times, but quite a bit fewer. 

So, similarly, of those 84 loops for tens, he only sometimes calls to evaluate hundreds. As mentioned, the set of 10 digits shrinks some more and this continues upward to functions that evaluate hundreds and thousands  and finally the one evaluating ten thousands pretty much prints out an answer it finds. 

So overall iterations can be shown to drop. We could add code to measure how many times each function is called and come up with an exact value for this built-in problem. I did and the functions were called this many times:

>>> counting
{'unit': 1, 'ten': 72, 'hundred': 290, 'thou': 183, '10thou': 196}
>>> sum(counting.values())
742

But I also see the permutation function was called 542 times. Most of those runs though were fairly trivial as the combined number of items issued back were only 7,390 as compared to nearly two million in the earlier version. Overall, this is much faster.

Range() is probably a highly efficient built-in written in C, but it can totally be eliminated in this code as it is a constant. You can write {1,2,2,3,4,5,6,7,8,9} or [0,1] instead of calculating ranges.

Naturally, this code does not scale up to finding the two solutions for:

HAWAII + IDAHO +IOWA + OHIO = STATES

The horizontal versions solved that easily.

The carry issues get more complex here even if a general solution is written. One approach might be to make a matrix as wide as the widest term and place all entries in it as characters, right justified. You can then work on any number of columns by extracting columns backwards from the right and applying whatever logic, perhaps recursively. The possible carry amounts need to be estimated as no more than about the number of items being added minus one.

But, I am ready to do other things so I leave it as an exercise for the reader ?

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Peter Otten
Sent: Friday, December 14, 2018 3:21 AM
To: python-list at python.org
Subject: Smarter algo, was Re: 03 digression by brute force

jfong at ms4.hinet.net wrote:

> Just for fun:-) On my ooold PC, it takes 0.047 seconds to run the 
> following algorithm on the problem 'SNED + MORE == MONEY".

> def tenThousand(u, Cin):  # Cin == M
>     global n
>     if Cin == M:
>         print(S, E, N, D, '+', M, O, R, E, '==', M, O, N, E, Y)
>         n += 1

And it probably took under two minutes with the brute force approach.
So if you were only interested in the result the code below were efficient only if it took at most two more minutes to write it;)

But seriously, the next step is to generalize this to work with arbitrary additions...
More fun!


-- 
https://mail.python.org/mailman/listinfo/python-list



From jfong at ms4.hinet.net  Sat Dec 15 04:56:26 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Sat, 15 Dec 2018 01:56:26 -0800 (PST)
Subject: Smarter algo, was Re: 03 digression by brute force
In-Reply-To: <mailman.52.1544832804.2771.python-list@python.org>
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
 <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>
 <puvp23$v6r$1@blaine.gmane.org>
 <012201d4940a$fb490f00$f1db2d00$@verizon.net>
 <mailman.52.1544832804.2771.python-list@python.org>
Message-ID: <9ddf88ae-6952-46fe-8f07-c76037ffbb7c@googlegroups.com>

Appreciate your thoughtfully analysis on this code. Before generalize it with arbitrary additions, as Peter suggested:-), a recursive version is needed. I may give it a try on this Sunday.


Avi Gross at 2018/12/15 UTC+8 AM8:13:37 wrote:
> REAL SUBJECT: Analysis of alternate algorithms.
> 
> Peter & Jach and anyone interested,
> 
> As Peter said in his altered subject line, Jack changed directions from tweaking an algorithm to trying something quite different.
> 
> Reminder of the problem. 
> 
> Horizontal View:
> SEND + MORE = MONEY
> 
> Vertical View:
>   SEND
> +MORE
> ...........
> MONEY
> 
> Hard to be precise as I am sticking to plain text in my message. The three words above are meant to be right adjusted.
> 
> When solving it horizontally, Jach and I used variants of a brute force approach. Substitute all possible combinations. He did it in-line and used eval. I did it by making lists of items on both sides and summing the int() of each component and comparing. We tried both our algorithms and his was slower and he profiled that the cause was that eval() was much more expensive as were his use of regular expression functions. For comparison, mine used int() and string manipulation functions and sets and dictionaries.
> 
> But the real puzzle is meant to be solved in a more vertical way by humans using logic. I won't do that here but note we have 4 equations going down but 8 unknowns. And the equations are not unique.
> 
> The rightmost column (I will call it the first as our arithmetic proceeds from right to left) is meant to represent ONES and provides the equation:
> 
> (D+E== Y) or (D+E == Y + 10)
> 
> Worse, for the next column, you either get a "1" carried from the previous addition or not and either pass a "1" along to the next column or not. 4 Possibilities.
> 
> (N+R==E) or (N+R+1==E) or (N+R==E+10) or (N+R+1==E+10)
> 
> Getting a headache yet?
> 
> For a human, they need a way to come up with additional info in terms of constraints.
> 
> There is a guiding inequality that looks like this:
> 
> S is not the same as any of the others. Anytime you solve for another, the list of possible values for S shrinks.
> Ditto for each other variable.
> Or, since anything plus 0 is itself, then D and E adding up to Y (with no possible carry) cannot be 0.
> 
> But getting a computer to do this might be a bit harder than blunt-force searches. So let's see why Jach's new algorithm was faster.
> 
> The code I am analyzing can be viewed in the archives and will not be entered again:
> 
> https://mail.python.org/pipermail/python-list/2018-December/738454.html
> 
> So what did Jach try in his newer version? It is what I call a vertical approach but one a computer can do easier than a human can or would. I see it is a very specific algorithm that hard codes in these variables as global entities that are altered by a set of nested functions. S, E, N, D, M, O, R, Y. There are approaches that might be better such as passing a dictionary partially filled out from one function to the next as the only one that prints the solution is the final function call.
> 
> So his is not a general solution.
> 
> What interests me as a probable reason this is faster is the number of situations it tries. The earlier versions asked itertools.permutations() to provide all unique combinations of ten tokens in eight positions. So there were 10 choices for the first and 9 for the second and so on adding up to 10!/2! or 1,814,400  different combinations tried. That approaches 2 million.
> 
> Jack broke the problem down into evaluating the units column with a loop like this:
> 
> itertools.permutations(range(10), 3)
> 
> That is 720 possibilities. He then doubled that to 1,440 to consider a carry. Only if the selected values for the three variables in contention (plus a carry) does he go on to call to evaluate the tens column.
> 
> It then shrinks a bit more as he no longer gets the permutations of all 10 digits. He subtracts the three values that might work for the units, so he is asking for permutations of 7 digits, two at a time. That is 42, doubled again to 84 for carry purposes. And this function is not called 1,440 times, but quite a bit fewer. 
> 
> So, similarly, of those 84 loops for tens, he only sometimes calls to evaluate hundreds. As mentioned, the set of 10 digits shrinks some more and this continues upward to functions that evaluate hundreds and thousands  and finally the one evaluating ten thousands pretty much prints out an answer it finds. 
> 
> So overall iterations can be shown to drop. We could add code to measure how many times each function is called and come up with an exact value for this built-in problem. I did and the functions were called this many times:
> 
> >>> counting
> {'unit': 1, 'ten': 72, 'hundred': 290, 'thou': 183, '10thou': 196}
> >>> sum(counting.values())
> 742
> 
> But I also see the permutation function was called 542 times. Most of those runs though were fairly trivial as the combined number of items issued back were only 7,390 as compared to nearly two million in the earlier version. Overall, this is much faster.
> 
> Range() is probably a highly efficient built-in written in C, but it can totally be eliminated in this code as it is a constant. You can write {1,2,2,3,4,5,6,7,8,9} or [0,1] instead of calculating ranges.
> 
> Naturally, this code does not scale up to finding the two solutions for:
> 
> HAWAII + IDAHO +IOWA + OHIO = STATES
> 
> The horizontal versions solved that easily.
> 
> The carry issues get more complex here even if a general solution is written. One approach might be to make a matrix as wide as the widest term and place all entries in it as characters, right justified. You can then work on any number of columns by extracting columns backwards from the right and applying whatever logic, perhaps recursively. The possible carry amounts need to be estimated as no more than about the number of items being added minus one.
> 
> But, I am ready to do other things so I leave it as an exercise for the reader ?
> 
> -----Original Message-----
> From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Peter Otten
> Sent: Friday, December 14, 2018 3:21 AM
> To: python-list at python.org
> Subject: Smarter algo, was Re: 03 digression by brute force
> 
> jfong at ms4.hinet.net wrote:
> 
> > Just for fun:-) On my ooold PC, it takes 0.047 seconds to run the 
> > following algorithm on the problem 'SNED + MORE == MONEY".
> 
> > def tenThousand(u, Cin):  # Cin == M
> >     global n
> >     if Cin == M:
> >         print(S, E, N, D, '+', M, O, R, E, '==', M, O, N, E, Y)
> >         n += 1
> 
> And it probably took under two minutes with the brute force approach.
> So if you were only interested in the result the code below were efficient only if it took at most two more minutes to write it;)
> 
> But seriously, the next step is to generalize this to work with arbitrary additions...
> More fun!
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list



From no-one at nowhere.org  Sat Dec 15 07:37:51 2018
From: no-one at nowhere.org (BlindAnagram)
Date: Sat, 15 Dec 2018 12:37:51 +0000
Subject: 03 digression by brute force
In-Reply-To: <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
 <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>
Message-ID: <3875f32d-fdd0-ac6c-30c0-521e19ea1c89@nowhere.org>

On 14/12/2018 02:24, jfong at ms4.hinet.net wrote:
> Just for fun:-) On my ooold PC, it takes 0.047 seconds to run the following algorithm on the problem 'SNED + MORE == MONEY".
> 
> -----------------------------
> import time
> import itertools
> 
> #S, E, N, D, M, O, R, Y
> n = 0
> digits = {x for x in range(10)}
> 
> def tenThousand(u, Cin):  # Cin == M
>     global n
>     if Cin == M:
>         print(S, E, N, D, '+', M, O, R, E, '==', M, O, N, E, Y)
>         n += 1
> 
> def thousand(u, Cin):  # Cin + S + M == 10 * Cout + O
>     global S, M, n
>     rest = digits - set(u)
>     for g in itertools.permutations(rest, 2):
>         for Cout in range(2):
>             if Cin + g[0] + g[1] == 10 * Cout + O:
>                 S = g[0];  M = g[1]
>                 tenThousand(u + g, Cout)
>             
> def hundred(u, Cin):  # Cin + E + O == 10 * Cout + N
>     global O, n
>     rest = digits - set(u)
>     for g in itertools.permutations(rest, 1):
>         for Cout in range(2):
>             if Cin + E + g[0] == 10 * Cout + N:
>                 O = g[0]
>                 thousand(u + g, Cout)
> 
> def ten(u, Cin):  # Cin + N + R == 10 * Cout + E
>     global N, R, n
>     rest = digits - set(u)
>     for g in itertools.permutations(rest, 2):
>         for Cout in range(2):
>             if Cin + g[0] + g[1] == 10 * Cout + E:
>                 N = g[0];  R = g[1]
>                 hundred(u + g, Cout)
>             
> def unit():  # D + E == 10 * Cout + Y
>     global D, E, Y, n
>     n = 0
>     for g in itertools.permutations(range(10), 3):  # g is a tuple
>         for Cout in range(2):  # add two items so Cout is 0 or 1
>             if g[0] + g[1] == 10 * Cout + g[2]:
>                 D = g[0];  E = g[1];  Y = g[2]
>                 ten(g, Cout)
>     print(n)
>         
> if __name__ == '__main__':
>     start = time.time()
>     unit()
>     print(time.time() - start)
> -------------------------
> 

There are quite a few Python based solvers for alphametics around on the
net, for example:

http://code.activestate.com/recipes/576615-alphametics-solver/

There is also a long discussion here on ways of doing this:

https://enigmaticcode.wordpress.com/2016/06/22/solving-alphametics-with-python/


From blindanagram at nowhere.com  Sat Dec 15 07:41:09 2018
From: blindanagram at nowhere.com (BlindAnagram)
Date: Sat, 15 Dec 2018 12:41:09 +0000
Subject: Smarter algo, was Re: 03 digression by brute force
In-Reply-To: <9ddf88ae-6952-46fe-8f07-c76037ffbb7c@googlegroups.com>
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
 <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>
 <puvp23$v6r$1@blaine.gmane.org> <012201d4940a$fb490f00$f1db2d00$@verizon.net>
 <mailman.52.1544832804.2771.python-list@python.org>
 <9ddf88ae-6952-46fe-8f07-c76037ffbb7c@googlegroups.com>
Message-ID: <Zs6dncM08ub-a4nBnZ2dnUU78IWdnZ2d@brightview.co.uk>

On 15/12/2018 09:56, jfong at ms4.hinet.net wrote:
> Appreciate your thoughtfully analysis on this code. Before generalize it with arbitrary additions, as Peter suggested:-), a recursive version is needed. I may give it a try on this Sunday.
> 
> 
> Avi Gross at 2018/12/15 UTC+8 AM8:13:37 wrote:
>> REAL SUBJECT: Analysis of alternate algorithms.
>>
>> Peter & Jach and anyone interested,
>>
>> As Peter said in his altered subject line, Jack changed directions from tweaking an algorithm to trying something quite different.
>>
>> Reminder of the problem. 
>>
>> Horizontal View:
>> SEND + MORE = MONEY
>>
>> Vertical View:
>>   SEND
>> +MORE
>> ...........
>> MONEY
>>
>> Hard to be precise as I am sticking to plain text in my message. The three words above are meant to be right adjusted.
>>
>> When solving it horizontally, Jach and I used variants of a brute force approach. Substitute all possible combinations. He did it in-line and used eval. I did it by making lists of items on both sides and summing the int() of each component and comparing. We tried both our algorithms and his was slower and he profiled that the cause was that eval() was much more expensive as were his use of regular expression functions. For comparison, mine used int() and string manipulation functions and sets and dictionaries.
>>
>> But the real puzzle is meant to be solved in a more vertical way by humans using logic. I won't do that here but note we have 4 equations going down but 8 unknowns. And the equations are not unique.
>>
>> The rightmost column (I will call it the first as our arithmetic proceeds from right to left) is meant to represent ONES and provides the equation:
>>
>> (D+E== Y) or (D+E == Y + 10)
>>
>> Worse, for the next column, you either get a "1" carried from the previous addition or not and either pass a "1" along to the next column or not. 4 Possibilities.
>>
>> (N+R==E) or (N+R+1==E) or (N+R==E+10) or (N+R+1==E+10)
>>
>> Getting a headache yet?
>>
>> For a human, they need a way to come up with additional info in terms of constraints.
>>
>> There is a guiding inequality that looks like this:
>>
>> S is not the same as any of the others. Anytime you solve for another, the list of possible values for S shrinks.
>> Ditto for each other variable.
>> Or, since anything plus 0 is itself, then D and E adding up to Y (with no possible carry) cannot be 0.
>>
>> But getting a computer to do this might be a bit harder than blunt-force searches. So let's see why Jach's new algorithm was faster.
>>
>> The code I am analyzing can be viewed in the archives and will not be entered again:
>>
>> https://mail.python.org/pipermail/python-list/2018-December/738454.html
>>
>> So what did Jach try in his newer version? It is what I call a vertical approach but one a computer can do easier than a human can or would. I see it is a very specific algorithm that hard codes in these variables as global entities that are altered by a set of nested functions. S, E, N, D, M, O, R, Y. There are approaches that might be better such as passing a dictionary partially filled out from one function to the next as the only one that prints the solution is the final function call.
>>
>> So his is not a general solution.
>>
>> What interests me as a probable reason this is faster is the number of situations it tries. The earlier versions asked itertools.permutations() to provide all unique combinations of ten tokens in eight positions. So there were 10 choices for the first and 9 for the second and so on adding up to 10!/2! or 1,814,400  different combinations tried. That approaches 2 million.
>>
>> Jack broke the problem down into evaluating the units column with a loop like this:
>>
>> itertools.permutations(range(10), 3)
>>
>> That is 720 possibilities. He then doubled that to 1,440 to consider a carry. Only if the selected values for the three variables in contention (plus a carry) does he go on to call to evaluate the tens column.
>>
>> It then shrinks a bit more as he no longer gets the permutations of all 10 digits. He subtracts the three values that might work for the units, so he is asking for permutations of 7 digits, two at a time. That is 42, doubled again to 84 for carry purposes. And this function is not called 1,440 times, but quite a bit fewer. 
>>
>> So, similarly, of those 84 loops for tens, he only sometimes calls to evaluate hundreds. As mentioned, the set of 10 digits shrinks some more and this continues upward to functions that evaluate hundreds and thousands  and finally the one evaluating ten thousands pretty much prints out an answer it finds. 
>>
>> So overall iterations can be shown to drop. We could add code to measure how many times each function is called and come up with an exact value for this built-in problem. I did and the functions were called this many times:
>>
>>>>> counting
>> {'unit': 1, 'ten': 72, 'hundred': 290, 'thou': 183, '10thou': 196}
>>>>> sum(counting.values())
>> 742
>>
>> But I also see the permutation function was called 542 times. Most of those runs though were fairly trivial as the combined number of items issued back were only 7,390 as compared to nearly two million in the earlier version. Overall, this is much faster.
>>
>> Range() is probably a highly efficient built-in written in C, but it can totally be eliminated in this code as it is a constant. You can write {1,2,2,3,4,5,6,7,8,9} or [0,1] instead of calculating ranges.
>>
>> Naturally, this code does not scale up to finding the two solutions for:
>>
>> HAWAII + IDAHO +IOWA + OHIO = STATES
>>
>> The horizontal versions solved that easily.
>>
>> The carry issues get more complex here even if a general solution is written. One approach might be to make a matrix as wide as the widest term and place all entries in it as characters, right justified. You can then work on any number of columns by extracting columns backwards from the right and applying whatever logic, perhaps recursively. The possible carry amounts need to be estimated as no more than about the number of items being added minus one.
>>
>> But, I am ready to do other things so I leave it as an exercise for the reader ?
>>
>> -----Original Message-----
>> From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Peter Otten
>> Sent: Friday, December 14, 2018 3:21 AM
>> To: python-list at python.org
>> Subject: Smarter algo, was Re: 03 digression by brute force
>>
>> jfong at ms4.hinet.net wrote:
>>
>>> Just for fun:-) On my ooold PC, it takes 0.047 seconds to run the 
>>> following algorithm on the problem 'SNED + MORE == MONEY".
>>
>>> def tenThousand(u, Cin):  # Cin == M
>>>     global n
>>>     if Cin == M:
>>>         print(S, E, N, D, '+', M, O, R, E, '==', M, O, N, E, Y)
>>>         n += 1
>>
>> And it probably took under two minutes with the brute force approach.
>> So if you were only interested in the result the code below were efficient only if it took at most two more minutes to write it;)
>>
>> But seriously, the next step is to generalize this to work with arbitrary additions...
>> More fun!

There are quite a few Python based solvers for alphametics around on the
net, for example:

http://code.activestate.com/recipes/576615-alphametics-solver/

There is also a long discussion here on ways of doing this:

https://enigmaticcode.wordpress.com/2016/06/22/solving-alphametics-with-python/


From avigross at verizon.net  Sat Dec 15 12:47:15 2018
From: avigross at verizon.net (Avi Gross)
Date: Sat, 15 Dec 2018 12:47:15 -0500
Subject: Smarter algo, was Re: 03 digression by brute force
In-Reply-To: <Zs6dncM08ub-a4nBnZ2dnUU78IWdnZ2d@brightview.co.uk>
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
 <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>
 <puvp23$v6r$1@blaine.gmane.org> <012201d4940a$fb490f00$f1db2d00$@verizon.net>
 <mailman.52.1544832804.2771.python-list@python.org>
 <9ddf88ae-6952-46fe-8f07-c76037ffbb7c@googlegroups.com>
 <Zs6dncM08ub-a4nBnZ2dnUU78IWdnZ2d@brightview.co.uk>
Message-ID: <007801d4949e$37c4b950$a74e2bf0$@verizon.net>

I appreciate the information by " BlindAnagram <blindanagram at nowhere.com>" below. I myself claim to be from erehwon at times.

But to be clear, there are issues raised here where someone wants an easy solution for the real world like "I have to build a webserver that searches a database" and they would prefer an answer telling them of a set of modules they can import that can do 90% of the work once you fill in some things on your own.

I am not clear on why Jach is working on this. Personally, I just like solving puzzles and consider it fun (in moderation) to think about the ideas behind a solution, alternates, ways to improve, and so on.

So  I wanted to balance the wasteful brute force aspects of a solution with ways to cut down the expense per iteration. In particular, I chose not to use the regular expression routines or eval. The first reference you supplied is a nice solution but does use those. It needs minor changes as it is written in an enhanced python 2.6 and I choose not to look back ?

And best, it offers an oodle of similar puzzles. The weirdest perhaps was this:

('AN + ACCELERATING + INFERENTIAL + ENGINEERING + TALE + ' +
            'ELITE + GRANT + FEE + ET + CETERA == ARTIFICIAL + INTELLIGENCE')

I mention that for a reason. In a private exchange with Jach, I noted that a generalization of his method would need to allow for more possible carries than just 0 and 1 that rise with the number of items being added. In the above, there are 10 numbers being added on the left and two on the right. Granted the rules do not allow all the letters to be 9.  So the maximum addition is not usually 90 + carry. But it can be in the 40's or even higher depending on the specific digits involved. In the worst case, you might have 10 copies in a column of the same letter whose value might be 9. So the algorithm would need to plan on a larger set of carry choices just in case. At some point, it might be less efficient than the horizontal solutions such as the ones Jach and I made first or the one you shared below.

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of BlindAnagram
Sent: Saturday, December 15, 2018 7:41 AM
To: python-list at python.org
Subject: Re: Smarter algo, was Re: 03 digression by brute force

<<<snipped>>>

There are quite a few Python based solvers for alphametics around on the net, for example:

http://code.activestate.com/recipes/576615-alphametics-solver/

There is also a long discussion here on ways of doing this:

https://enigmaticcode.wordpress.com/2016/06/22/solving-alphametics-with-python/




From hmjupite at aggies.ncat.edu  Sat Dec 15 17:56:36 2018
From: hmjupite at aggies.ncat.edu (Holley Jupiter)
Date: Sat, 15 Dec 2018 17:56:36 -0500
Subject: No module named encodings
Message-ID: <5c1586a6.1c69fb81.596ee.7fc7@mx.google.com>

Having an issue with this error. I have uninstalled and reinstalled several times but I cannot solve the problem. I googled my issue but non of the solutions have worked.

Sent from Mail for Windows 10



From marc at marcsnet.com  Sat Dec 15 19:55:11 2018
From: marc at marcsnet.com (Marc Lucke)
Date: Sun, 16 Dec 2018 11:55:11 +1100
Subject: clusters of numbers
Message-ID: <ab5ef5ef-f046-7de9-ce80-e0bce6d0aae1@marcsnet.com>

hey guys,

I have a hobby project that sorts my email automatically for me & I want 
to improve it.? There's data science and statistical info that I'm 
missing, & I always enjoy reading about the pythonic way to do things too.

I have a list of percentage scores:

(1,11,1,7,5,7,2,2,2,10,10,1,2,2,1,7,2,1,7,5,3,8,2,6,3,2,7,2,12,3,1,2,19,3,5,1,1,7,8,8,1,5,6,7,3,14,6,1,6,7,6,15,6,3,7,2,6,23,2,7,1,21,21,8,8,3,2,20,1,3,12,3,1,2,10,16,16,15,6,5,3,2,2,11,1,14,6,3,7,1,5,3,3,14,3,7,3,5,8,3,6,17,1,1,7,3,1,2,6,1,7,7,12,6,6,2,1,6,3,6,2,1,5,1,8,10,2,6,1,7,3,5,7,7,5,7,2,5,1,19,19,1,12,5,10,2,19,1,3,19,6,1,5,11,2,1,2,5,2,5,8,2,2,2,5,3,1,21,2,3,7,10,1,8,1,3,17,17,1,5,3,10,14,1,2,14,14,1,15,6,3,2,17,17,1,1,1,2,2,3,3,2,2,7,7,2,1,2,8,2,20,3,2,3,12,7,6,5,12,2,3,11,3,1,1,8,16,10,1,6,6,6,11,1,6,5,2,5,11,1,2,10,6,14,6,3,3,5,2,6,17,15,1,2,2,17,5,3,3,5,8,1,6,3,14,3,2,1,7,2,8,11,5,14,3,19,1,3,7,3,3,8,8,6,1,3,1,14,14,10,3,2,1,12,2,3,1,2,2,6,6,7,10,10,12,24,1,21,21,5,11,12,12,2,1,19,8,6,2,1,1,19,10,6,2,15,15,7,10,14,12,14,5,11,7,12,2,1,14,10,7,10,3,17,25,10,5,5,3,12,5,2,14,5,8,1,11,5,29,2,7,20,12,14,1,10,6,17,16,6,7,11,12,3,1,23,11,10,11,5,10,6,2,17,15,20,5,10,1,17,3,7,15,5,11,6,19,14,15,7,1,2,17,8,15,10,26,6,1,2,10,6,14,12,6,1,16,6,12,10,10,14,1,6,1,6,6,12,6,6,1,2,5,10,8,10,1,6,8,17,11,6,3,6,5,1,2,1,2,6,6,12,14,7,1,7,1,8,2,3,14,11,6,3,11,3,1,6,17,12,8,2,10,3,12,12,2,7,5,5,17,2,5,10,12,21,15,6,10,10,7,15,11,2,7,10,3,1,2,7,10,15,1,1,6,5,5,3,17,19,7,1,15,2,8,7,1,6,2,1,15,19,7,15,1,8,3,3,20,8,1,11,7,8,7,1,12,11,1,10,17,2,23,3,7,20,20,3,11,5,1,1,8,1,6,2,11,1,5,1,10,7,20,17,8,1,2,10,6,2,1,23,11,11,7,2,21,5,5,8,1,1,10,12,15,2,1,10,5,2,2,5,1,2,11,10,1,8,10,12,2,12,2,8,6,19,15,8,2,16,7,5,14,2,1,3,3,10,16,20,5,8,14,8,3,14,2,1,5,16,16,2,10,8,17,17,10,10,11,3,5,1,17,17,3,17,5,6,7,7,12,19,15,20,11,10,2,6,6,5,5,1,16,16,8,7,2,1,3,5,20,20,6,7,5,23,14,3,10,2,2,7,10,10,3,5,5,8,14,11,14,14,11,19,5,5,2,12,25,5,2,11,8,10,5,11,10,12,10,2,15,15,15,5,10,1,12,14,8,5,6,2,26,15,21,15,12,2,8,11,5,5,16,5,2,17,3,2,2,3,15,3,8,10,7,10,3,1,14,14,8,8,8,19,10,12,3,8,2,20,16,10,6,15,6,1,12,12,15,15,8,11,17,7,7,7,3,10,1,5,19,11,7,12,8,12,7,5,10,1,11,1,6,21,1,1,10,3,8,5,6,5,20,25,17,5,2,16,14,11,1,17,10,14,5,16,5,2,7,3,8,17,7,19,12,6,5,1,3,12,43,11,8,11,5,19,10,5,11,7,20,6,12,35,5,3,17,10,2,12,6,5,21,24,15,5,10,3,15,1,12,6,3,17,3,2,3,5,5,14,11,8,1,8,10,5,25,8,7,2,6,3,11,1,11,7,3,10,7,12,10,8,6,1,1,17,3,1,1,2,19,6,10,2,2,7,5,16,3,2,11,10,7,10,21,3,5,2,21,3,14,6,7,2,24,3,17,3,21,8,5,11,17,5,6,10,5,20,1,12,2,3,20,6,11,12,14,6,6,1,14,15,12,15,6,20,7,7,19,3,7,5,16,12,6,7,2,10,3,2,11,8,6,6,5,1,11,1,15,21,14,6,3,2,2,5,6,1,3,5,3,6,20,1,15,12,2,3,3,7,1,16,5,24,10,7,1,12,16,8,26,16,15,10,19,11,6,6,5,6,5)

 ?& I'd like to know know whether, & how the numbers are clustered.? In 
an extreme & illustrative example, 1..10 would have zero clusters;? 
1,1,1,2,2,2,7,7,7 would have 3 clusters (around 1,2 & 7); 
17,22,20,45,47,51,82,84,83? would have 3 clusters. (around 20, 47 & 
83).? In my set, when I scan it, I intuitively figure there's lots of 
numbers close to 0 & a lot close to 20 (or there abouts).

I saw info about k-clusters but I'm not sure if I'm going down the right 
path.? I'm interested in k-clusters & will teach myself, but my priority 
is working out this problem.

Do you know the name of the algorithm I'm trying to use?? If so, are 
there python libraries like numpy that I can leverage?? I imagine that I 
could iterate from 0 to 100% using that as an artificial mean, discard 
values that are over a standard deviation away, and count the number of 
scores for that mean; then at the end of that I could set a threshold 
for which the artificial mean would be kept something like (no attempt 
at correct syntax:

means={}
deviation=5
threshold=int(0.25*len(list))
for i in range 100:
 ? count=0
 ? for j in list:
 ??? if abs(j-i) > deviation:
 ????? count+=1
 ? if count > threshold:
 ??? means[i]=count

That algorithm is entirely untested & I think it could work, it's just I 
don't want to reinvent the wheel.? Any ideas kindly appreciated.




From oscar.j.benjamin at gmail.com  Sat Dec 15 21:18:53 2018
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sun, 16 Dec 2018 02:18:53 +0000
Subject: clusters of numbers
In-Reply-To: <ab5ef5ef-f046-7de9-ce80-e0bce6d0aae1@marcsnet.com>
References: <ab5ef5ef-f046-7de9-ce80-e0bce6d0aae1@marcsnet.com>
Message-ID: <CAHVvXxQozWmWLjXhvSByk-C9_fzfFtPvrPwgWXVUTeZYJPYZ8w@mail.gmail.com>

On Sun, 16 Dec 2018 at 01:47, Marc Lucke <marc at marcsnet.com> wrote:
>
> hey guys,
>
> I have a hobby project that sorts my email automatically for me & I want
> to improve it.  There's data science and statistical info that I'm
> missing, & I always enjoy reading about the pythonic way to do things too.
>
> I have a list of percentage scores:
>
> (1,11,1,7,5,7,2,2,2,10,10,1,2,2,1,7,2,1,7,5,3,8,2,6,3,2,7,2,12,3,1,2,19,3,5,1,1,7,8,8,1,5,6,7,3,14,6,1,6,7,6,15,6,3,7,2,6,23,2,7,1,21,21,8,8,3,2,20,1,3,12,3,1,2,10,16,16,15,6,5,3,2,2,11,1,14,6,3,7,1,5,3,3,14,3,7,3,5,8,3,6,17,1,1,7,3,1,2,6,1,7,7,12,6,6,2,1,6,3,6,2,1,5,1,8,10,2,6,1,7,3,5,7,7,5,7,2,5,1,19,19,1,12,5,10,2,19,1,3,19,6,1,5,11,2,1,2,5,2,5,8,2,2,2,5,3,1,21,2,3,7,10,1,8,1,3,17,17,1,5,3,10,14,1,2,14,14,1,15,6,3,2,17,17,1,1,1,2,2,3,3,2,2,7,7,2,1,2,8,2,20,3,2,3,12,7,6,5,12,2,3,11,3,1,1,8,16,10,1,6,6,6,11,1,6,5,2,5,11,1,2,10,6,14,6,3,3,5,2,6,17,15,1,2,2,17,5,3,3,5,8,1,6,3,14,3,2,1,7,2,8,11,5,14,3,19,1,3,7,3,3,8,8,6,1,3,1,14,14,10,3,2,1,12,2,3,1,2,2,6,6,7,10,10,12,24,1,21,21,5,11,12,12,2,1,19,8,6,2,1,1,19,10,6,2,15,15,7,10,14,12,14,5,11,7,12,2,1,14,10,7,10,3,17,25,10,5,5,3,12,5,2,14,5,8,1,11,5,29,2,7,20,12,14,1,10,6,17,16,6,7,11,12,3,1,23,11,10,11,5,10,6,2,17,15,20,5,10,1,17,3,7,15,5,11,6,19,14,15,7,1,2,17,8,15,10,26,6,1,2,10,6,14,12,6,1,16,6,12,10,10,14,1,6,1,6,6,12,6,6,1,2,5,10,8,10,1,6,8,17,11,6,3,6,5,1,2,1,2,6,6,12,14,7,1,7,1,8,2,3,14,11,6,3,11,3,1,6,17,12,8,2,10,3,12,12,2,7,5,5,17,2,5,10,12,21,15,6,10,10,7,15,11,2,7,10,3,1,2,7,10,15,1,1,6,5,5,3,17,19,7,1,15,2,8,7,1,6,2,1,15,19,7,15,1,8,3,3,20,8,1,11,7,8,7,1,12,11,1,10,17,2,23,3,7,20,20,3,11,5,1,1,8,1,6,2,11,1,5,1,10,7,20,17,8,1,2,10,6,2,1,23,11,11,7,2,21,5,5,8,1,1,10,12,15,2,1,10,5,2,2,5,1,2,11,10,1,8,10,12,2,12,2,8,6,19,15,8,2,16,7,5,14,2,1,3,3,10,16,20,5,8,14,8,3,14,2,1,5,16,16,2,10,8,17,17,10,10,11,3,5,1,17,17,3,17,5,6,7,7,12,19,15,20,11,10,2,6,6,5,5,1,16,16,8,7,2,1,3,5,20,20,6,7,5,23,14,3,10,2,2,7,10,10,3,5,5,8,14,11,14,14,11,19,5,5,2,12,25,5,2,11,8,10,5,11,10,12,10,2,15,15,15,5,10,1,12,14,8,5,6,2,26,15,21,15,12,2,8,11,5,5,16,5,2,17,3,2,2,3,15,3,8,10,7,10,3,1,14,14,8,8,8,19,10,12,3,8,2,20,16,10,6,15,6,1,12,12,15,15,8,11,17,7,7,7,3,10,1,5,19,11,7,12,8,12,7,5,10,1,11,1,6,21,1,1,10,3,8,5,6,5,20,25,17,5,2,16,14,11,1,17,10,14,5,16,5,2,7,3,8,17,7,19,12,6,5,1,3,12,43,11,8,11,5,19,10,5,11,7,20,6,12,35,5,3,17,10,2,12,6,5,21,24,15,5,10,3,15,1,12,6,3,17,3,2,3,5,5,14,11,8,1,8,10,5,25,8,7,2,6,3,11,1,11,7,3,10,7,12,10,8,6,1,1,17,3,1,1,2,19,6,10,2,2,7,5,16,3,2,11,10,7,10,21,3,5,2,21,3,14,6,7,2,24,3,17,3,21,8,5,11,17,5,6,10,5,20,1,12,2,3,20,6,11,12,14,6,6,1,14,15,12,15,6,20,7,7,19,3,7,5,16,12,6,7,2,10,3,2,11,8,6,6,5,1,11,1,15,21,14,6,3,2,2,5,6,1,3,5,3,6,20,1,15,12,2,3,3,7,1,16,5,24,10,7,1,12,16,8,26,16,15,10,19,11,6,6,5,6,5)
>
>   & I'd like to know know whether, & how the numbers are clustered.  In
> an extreme & illustrative example, 1..10 would have zero clusters;
> 1,1,1,2,2,2,7,7,7 would have 3 clusters (around 1,2 & 7);
> 17,22,20,45,47,51,82,84,83  would have 3 clusters. (around 20, 47 &
> 83).  In my set, when I scan it, I intuitively figure there's lots of
> numbers close to 0 & a lot close to 20 (or there abouts).
>
> I saw info about k-clusters but I'm not sure if I'm going down the right
> path.  I'm interested in k-clusters & will teach myself, but my priority
> is working out this problem.
>
> Do you know the name of the algorithm I'm trying to use?

I don't recognise the algorithm in your code but when you say
k-clusters I assume you mean k-means:
https://en.wikipedia.org/wiki/K-means_clustering

Most discussions of k-means will assume that you are working in at
least 2 dimensions but in your case your data is 1D so bear that in
mind when comparing.

It's not hard to implement k-means yourself but I believe that
scikit-learn already has it:
https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html

--
Oscar


From vincent at vincentdavis.net  Sat Dec 15 23:15:30 2018
From: vincent at vincentdavis.net (Vincent Davis)
Date: Sat, 15 Dec 2018 21:15:30 -0700
Subject: clusters of numbers
In-Reply-To: <ab5ef5ef-f046-7de9-ce80-e0bce6d0aae1@marcsnet.com>
References: <ab5ef5ef-f046-7de9-ce80-e0bce6d0aae1@marcsnet.com>
Message-ID: <CALyJZZU1=hvQuxAzg9GhRSgn6GbY4i7WJQfp_q1XsuS3=O6YDQ@mail.gmail.com>

Why not start with a histogram.

Vincent

On Sat, Dec 15, 2018 at 6:46 PM Marc Lucke <marc at marcsnet.com> wrote:

> hey guys,
>
> I have a hobby project that sorts my email automatically for me & I want
> to improve it.  There's data science and statistical info that I'm
> missing, & I always enjoy reading about the pythonic way to do things too.
>
> I have a list of percentage scores:
>
>
> (1,11,1,7,5,7,2,2,2,10,10,1,2,2,1,7,2,1,7,5,3,8,2,6,3,2,7,2,12,3,1,2,19,3,5,1,1,7,8,8,1,5,6,7,3,14,6,1,6,7,6,15,6,3,7,2,6,23,2,7,1,21,21,8,8,3,2,20,1,3,12,3,1,2,10,16,16,15,6,5,3,2,2,11,1,14,6,3,7,1,5,3,3,14,3,7,3,5,8,3,6,17,1,1,7,3,1,2,6,1,7,7,12,6,6,2,1,6,3,6,2,1,5,1,8,10,2,6,1,7,3,5,7,7,5,7,2,5,1,19,19,1,12,5,10,2,19,1,3,19,6,1,5,11,2,1,2,5,2,5,8,2,2,2,5,3,1,21,2,3,7,10,1,8,1,3,17,17,1,5,3,10,14,1,2,14,14,1,15,6,3,2,17,17,1,1,1,2,2,3,3,2,2,7,7,2,1,2,8,2,20,3,2,3,12,7,6,5,12,2,3,11,3,1,1,8,16,10,1,6,6,6,11,1,6,5,2,5,11,1,2,10,6,14,6,3,3,5,2,6,17,15,1,2,2,17,5,3,3,5,8,1,6,3,14,3,2,1,7,2,8,11,5,14,3,19,1,3,7,3,3,8,8,6,1,3,1,14,14,10,3,2,1,12,2,3,1,2,2,6,6,7,10,10,12,24,1,21,21,5,11,12,12,2,1,19,8,6,2,1,1,19,10,6,2,15,15,7,10,14,12,14,5,11,7,12,2,1,14,10,7,10,3,17,25,10,5,5,3,12,5,2,14,5,8,1,11,5,29,2,7,20,12,14,1,10,6,17,16,6,7,11,12,3,1,23,11,10,11,5,10,6,2,17,15,20,5,10,1,17,3,7,15,5,11,6,19,14,15,7,1,2,17,8,15,10,26,6,1,2,10,6,14,12,6,1,16,6,12,10,10,14,1,6,1,6,6,12,6,6,1,2,5,10,8,10,1,6,8,17,11,6,3,6,5,1,2,1,2,6,6,12,14,7,1,7,1,8,2,3,14,11,6,3,11,3,1,6,17,12,8,2,10,3,12,12,2,7,5,5,17,2,5,10,12,21,15,6,10,10,7,15,11,2,7,10,3,1,2,7,10,15,1,1,6,5,5,3,17,19,7,1,15,2,8,7,1,6,2,1,15,19,7,15,1,8,3,3,20,8,1,11,7,8,7,1,12,11,1,10,17,2,23,3,7,20,20,3,11,5,1,1,8,1,6,2,11,1,5,1,10,7,20,17,8,1,2,10,6,2,1,23,11,11,7,2,21,5,5,8,1,1,10,12,15,2,1,10,5,2,2,5,1,2,11,10,1,8,10,12,2,12,2,8,6,19,15,8,2,16,7,5,14,2,1,3,3,10,16,20,5,8,14,8,3,14,2,1,5,16,16,2,10,8,17,17,10,10,11,3,5,1,17,17,3,17,5,6,7,7,12,19,15,20,11,10,2,6,6,5,5,1,16,16,8,7,2,1,3,5,20,20,6,7,5,23,14,3,10,2,2,7,10,10,3,5,5,8,14,11,14,14,11,19,5,5,2,12,25,5,2,11,8,10,5,11,10,12,10,2,15,15,15,5,10,1,12,14,8,5,6,2,26,15,21,15,12,2,8,11,5,5,16,5,2,17,3,2,2,3,15,3,8,10,7,10,3,1,14,14,8,8,8,19,10,12,3,8,2,20,16,10,6,15,6,1,12,12,15,15,8,11,17,7,7,7,3,10,1,5,19,11,7,12,8,12,7,5,10,1,11,1,6,21,1,1,10,3,8,5,6,5,20,25,17,5,2,16,14,11,1,17,10,14,5,16,5,2,7,3,8,17,7,19,12,6,5,1,3,12,43,11,8,11,5,19,10,5,11,7,20,6,12,35,5,3,17,10,2,12,6,5,21,24,15,5,10,3,15,1,12,6,3,17,3,2,3,5,5,14,11,8,1,8,10,5,25,8,7,2,6,3,11,1,11,7,3,10,7,12,10,8,6,1,1,17,3,1,1,2,19,6,10,2,2,7,5,16,3,2,11,10,7,10,21,3,5,2,21,3,14,6,7,2,24,3,17,3,21,8,5,11,17,5,6,10,5,20,1,12,2,3,20,6,11,12,14,6,6,1,14,15,12,15,6,20,7,7,19,3,7,5,16,12,6,7,2,10,3,2,11,8,6,6,5,1,11,1,15,21,14,6,3,2,2,5,6,1,3,5,3,6,20,1,15,12,2,3,3,7,1,16,5,24,10,7,1,12,16,8,26,16,15,10,19,11,6,6,5,6,5)
>
>   & I'd like to know know whether, & how the numbers are clustered.  In
> an extreme & illustrative example, 1..10 would have zero clusters;
> 1,1,1,2,2,2,7,7,7 would have 3 clusters (around 1,2 & 7);
> 17,22,20,45,47,51,82,84,83  would have 3 clusters. (around 20, 47 &
> 83).  In my set, when I scan it, I intuitively figure there's lots of
> numbers close to 0 & a lot close to 20 (or there abouts).
>
> I saw info about k-clusters but I'm not sure if I'm going down the right
> path.  I'm interested in k-clusters & will teach myself, but my priority
> is working out this problem.
>
> Do you know the name of the algorithm I'm trying to use?  If so, are
> there python libraries like numpy that I can leverage?  I imagine that I
> could iterate from 0 to 100% using that as an artificial mean, discard
> values that are over a standard deviation away, and count the number of
> scores for that mean; then at the end of that I could set a threshold
> for which the artificial mean would be kept something like (no attempt
> at correct syntax:
>
> means={}
> deviation=5
> threshold=int(0.25*len(list))
> for i in range 100:
>    count=0
>    for j in list:
>      if abs(j-i) > deviation:
>        count+=1
>    if count > threshold:
>      means[i]=count
>
> That algorithm is entirely untested & I think it could work, it's just I
> don't want to reinvent the wheel.  Any ideas kindly appreciated.
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>


From avigross at verizon.net  Sat Dec 15 23:43:44 2018
From: avigross at verizon.net (Avi Gross)
Date: Sat, 15 Dec 2018 23:43:44 -0500
Subject: clusters of numbers
References: <ab5ef5ef-f046-7de9-ce80-e0bce6d0aae1@marcsnet.com> 
Message-ID: <002901d494f9$ed6b0b50$c84121f0$@verizon.net>



-----Original Message-----
From: Avi Gross <avigross at verizon.net> 
Sent: Saturday, December 15, 2018 11:27 PM
To: 'Marc Lucke' <marc at marcsnet.com>
Subject: RE: clusters of numbers

Marc,

There are k-means implementations in python, R and other places. Most uses would have two or more dimensions with a goal of specifying how many clusters to look for and then it iterates starting with random existing points to cluster things near those points and then near the centers of those clusters until things stabilize.

Your data is 1-D. Something simpler like a bar chart makes sense. But that may not show underlying patterns.

I am more familiar with doing graphics in R but you can see a tabular view of your data:

data
  1   2   3   5   6   7   8  10  11  12  14  15  16  17  19  20  21  23  24  25  26  29  35  43 
124 116  97  95  89  74  57  73  48  49  38  35  20  33  21  19  14   5   4   4   3   1   1   1

There are clear gaps and a bar chart (which I cannot attach but could send in private email) does show clusters visibly.

But those may largely be an artifact of the missing info.

If you tell us more, we might be able to provide a better statistical answer. I assume you know how to get means and so on.

   vars    n mean   sd median trimmed  mad min max range skew kurtosis   se
X1    1 1021 7.82 6.01      6    7.12 5.93   1  43    42 1.04     1.23 0.19

Yes, the above is hard to read as I cannot use tables or a constant width font in this forum.

I ran a kmeans asking for 3 clusters:

1 16.512097
2  1.919881
3  7.433486

The three clusters had these scores in them:

Cluster 1: 5  6  7  8 10 11
Cluster 2:  1 2 3
Cluster 3: 12 14 15 16 17 19 20 21 23 24 25 26 29 35 43

If I run it asking for say 5 clusters:

Centers:

1  6.295238
2 11.432692
3  1.483333
4  3.000000
5 18.478261

And here are your five clusters:

5 6 7 8
10 11 12 14
1 2
3
15 16 17 19 20 21 23 24 25 26 29 35 43

If you ran this for various numbers, you might see one that makes more sense to you.  Or, maybe not.

We culd tell you what functions to use but if you search using keywords like python (or another language) followed by k-means or kmeans you can fid out what to install and use. In python, you would need Numpy and probably SciPy as well as the sklearn modules with the Kmeans function in sklearn.clusters. Note you can fine tune the algorithm multiple ways or run it several times as the results can depend on the initial guesses. And you may want to be able to make graphics showing the clusters, albeit it is 1-D.

Good luck.


-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Marc Lucke
Sent: Saturday, December 15, 2018 7:55 PM
To: python-list at python.org
Subject: clusters of numbers

hey guys,

I have a hobby project that sorts my email automatically for me & I want to improve it.  There's data science and statistical info that I'm missing, & I always enjoy reading about the pythonic way to do things too.

I have a list of percentage scores:

(1,11,1,7,5,7,2,2,2,10,10,1,2,2,1,7,2,1,7,5,3,8,2,6,3,2,7,2,12,3,1,2,19,3,5,1,1,7,8,8,1,5,6,7,3,14,6,1,6,7,6,15,6,3,7,2,6,23,2,7,1,21,21,8,8,3,2,20,1,3,12,3,1,2,10,16,16,15,6,5,3,2,2,11,1,14,6,3,7,1,5,3,3,14,3,7,3,5,8,3,6,17,1,1,7,3,1,2,6,1,7,7,12,6,6,2,1,6,3,6,2,1,5,1,8,10,2,6,1,7,3,5,7,7,5,7,2,5,1,19,19,1,12,5,10,2,19,1,3,19,6,1,5,11,2,1,2,5,2,5,8,2,2,2,5,3,1,21,2,3,7,10,1,8,1,3,17,17,1,5,3,10,14,1,2,14,14,1,15,6,3,2,17,17,1,1,1,2,2,3,3,2,2,7,7,2,1,2,8,2,20,3,2,3,12,7,6,5,12,2,3,11,3,1,1,8,16,10,1,6,6,6,11,1,6,5,2,5,11,1,2,10,6,14,6,3,3,5,2,6,17,15,1,2,2,17,5,3,3,5,8,1,6,3,14,3,2,1,7,2,8,11,5,14,3,19,1,3,7,3,3,8,8,6,1,3,1,14,14,10,3,2,1,12,2,3,1,2,2,6,6,7,10,10,12,24,1,21,21,5,11,12,12,2,1,19,8,6,2,1,1,19,10,6,2,15,15,7,10,14,12,14,5,11,7,12,2,1,14,10,7,10,3,17,25,10,5,5,3,12,5,2,14,5,8,1,11,5,29,2,7,20,12,14,1,10,6,17,16,6,7,11,12,3,1,23,11,10,11,5,10,6,2,17,15,20,5,10,1,17,3,7,15,5,11,6,19,14,15,7,1,2,17,8,15,10,26,6,1,2,10,6,14,12,6,1,16,6,12,10,10,14,1,6,1,6,6,12,6,6,1,2,5,10,8,10,1,6,8,17,11,6,3,6,5,1,2,1,2,6,6,12,14,7,1,7,1,8,2,3,14,11,6,3,11,3,1,6,17,12,8,2,10,3,12,12,2,7,5,5,17,2,5,10,12,21,15,6,10,10,7,15,11,2,7,10,3,1,2,7,10,15,1,1,6,5,5,3,17,19,7,1,15,2,8,7,1,6,2,1,15,19,7,15,1,8,3,3,20,8,1,11,7,8,7,1,12,11,1,10,17,2,23,3,7,20,20,3,11,5,1,1,8,1,6,2,11,1,5,1,10,7,20,17,8,1,2,10,6,2,1,23,11,11,7,2,21,5,5,8,1,1,10,12,15,2,1,10,5,2,2,5,1,2,11,10,1,8,10,12,2,12,2,8,6,19,15,8,2,16,7,5,14,2,1,3,3,10,16,20,5,8,14,8,3,14,2,1,5,16,16,2,10,8,17,17,10,10,11,3,5,1,17,17,3,17,5,6,7,7,12,19,15,20,11,10,2,6,6,5,5,1,16,16,8,7,2,1,3,5,20,20,6,7,5,23,14,3,10,2,2,7,10,10,3,5,5,8,14,11,14,14,11,19,5,5,2,12,25,5,2,11,8,10,5,11,10,12,10,2,15,15,15,5,10,1,12,14,8,5,6,2,26,15,21,15,12,2,8,11,5,5,16,5,2,17,3,2,2,3,15,3,8,10,7,10,3,1,14,14,8,8,8,19,10,12,3,8,2,20,16,10,6,15,6,1,12,12,15,15,8,11,17,7,7,7,3,10,1,5,19,11,7,12,8,12,7,5,10,1,11,1,6,21,1,1,10,3,8,5,6,5,20,25,17,5,2,16,14,11,1,17,10,14,5,16,5,2,7,3,8,17,7,19,12,6,5,1,3,12,43,11,8,11,5,19,10,5,11,7,20,6,12,35,5,3,17,10,2,12,6,5,21,24,15,5,10,3,15,1,12,6,3,17,3,2,3,5,5,14,11,8,1,8,10,5,25,8,7,2,6,3,11,1,11,7,3,10,7,12,10,8,6,1,1,17,3,1,1,2,19,6,10,2,2,7,5,16,3,2,11,10,7,10,21,3,5,2,21,3,14,6,7,2,24,3,17,3,21,8,5,11,17,5,6,10,5,20,1,12,2,3,20,6,11,12,14,6,6,1,14,15,12,15,6,20,7,7,19,3,7,5,16,12,6,7,2,10,3,2,11,8,6,6,5,1,11,1,15,21,14,6,3,2,2,5,6,1,3,5,3,6,20,1,15,12,2,3,3,7,1,16,5,24,10,7,1,12,16,8,26,16,15,10,19,11,6,6,5,6,5)

  & I'd like to know know whether, & how the numbers are clustered.  In an extreme & illustrative example, 1..10 would have zero clusters;
1,1,1,2,2,2,7,7,7 would have 3 clusters (around 1,2 & 7);
17,22,20,45,47,51,82,84,83  would have 3 clusters. (around 20, 47 & 83).  In my set, when I scan it, I intuitively figure there's lots of numbers close to 0 & a lot close to 20 (or there abouts).

I saw info about k-clusters but I'm not sure if I'm going down the right path.  I'm interested in k-clusters & will teach myself, but my priority is working out this problem.

Do you know the name of the algorithm I'm trying to use?  If so, are there python libraries like numpy that I can leverage?  I imagine that I could iterate from 0 to 100% using that as an artificial mean, discard values that are over a standard deviation away, and count the number of scores for that mean; then at the end of that I could set a threshold for which the artificial mean would be kept something like (no attempt at correct syntax:

means={}
deviation=5
threshold=int(0.25*len(list))
for i in range 100:
   count=0
   for j in list:
     if abs(j-i) > deviation:
       count+=1
   if count > threshold:
     means[i]=count

That algorithm is entirely untested & I think it could work, it's just I don't want to reinvent the wheel.  Any ideas kindly appreciated.


-- 
https://mail.python.org/mailman/listinfo/python-list



From shakti.shrivastava13 at gmail.com  Sat Dec 15 23:56:11 2018
From: shakti.shrivastava13 at gmail.com (Shakti Kumar)
Date: Sun, 16 Dec 2018 10:26:11 +0530
Subject: clusters of numbers
In-Reply-To: <CALyJZZU1=hvQuxAzg9GhRSgn6GbY4i7WJQfp_q1XsuS3=O6YDQ@mail.gmail.com>
References: <ab5ef5ef-f046-7de9-ce80-e0bce6d0aae1@marcsnet.com>
 <CALyJZZU1=hvQuxAzg9GhRSgn6GbY4i7WJQfp_q1XsuS3=O6YDQ@mail.gmail.com>
Message-ID: <CABi+06B_q6v7vgNxAw+uCX8tkM18QUWP3ZpHVQ0cnZFHTVyEsA@mail.gmail.com>

On Sun, 16 Dec 2018 at 09:49, Vincent Davis <vincent at vincentdavis.net> wrote:
>
> Why not start with a histogram.
>
> Vincent
>
> On Sat, Dec 15, 2018 at 6:46 PM Marc Lucke <marc at marcsnet.com> wrote:
>
> > hey guys,
> >
> > I have a hobby project that sorts my email automatically for me & I want
> > to improve it.  There's data science and statistical info that I'm
> > missing, & I always enjoy reading about the pythonic way to do things too.
> >
> > I have a list of percentage scores:

[clipped for brevity]

> > That algorithm is entirely untested & I think it could work, it's just I
> > don't want to reinvent the wheel.  Any ideas kindly appreciated.
> >
> >
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
> --
> https://mail.python.org/mailman/listinfo/python-list

+1 for k means certainly.
Also k means in 1D will be like a simple distance comparison and
assignment. A quick Google will give you the exact codes for doing so.
It will be you yourself who will decide how many clusters you want, as
Avi has rightly pointed out.


--
/Shakti


From ikorot01 at gmail.com  Sun Dec 16 01:58:33 2018
From: ikorot01 at gmail.com (Igor Korot)
Date: Sun, 16 Dec 2018 00:58:33 -0600
Subject: No connection
In-Reply-To: <CAKgKVtwCMf+LN5OsUKeQpkZC7ea9h=tuPOOwdcv-LoRRAcu8Pg@mail.gmail.com>
References: <CAKgKVtwCMf+LN5OsUKeQpkZC7ea9h=tuPOOwdcv-LoRRAcu8Pg@mail.gmail.com>
Message-ID: <CA+FnnTwkbYJQ-4k1OWmUd75i_1UOEO3tkKbhHM=wQAquqxhfaw@mail.gmail.com>

Hi,

On Sun, Dec 16, 2018 at 12:00 PM Vasilis Mytilinaios
<mytilinaiosvasilis at gmail.com> wrote:
>
> Hello,
>
> I'm trying to open the interactive shell but it doesn't let me. It says
> that IDLE's subprocess didn't make connection. I unistall and install
> python again and still doesn't work.
> I was using it properly until 2 days ago.
> Any advice on how can I fix it?

What is your version of python?
Did you upgrade it recently?
Was there an OS update recently applied?

Thank you.

>
> Thank you,
> Vasilis
> --
> https://mail.python.org/mailman/listinfo/python-list


From chbailly at gmail.com  Sun Dec 16 03:21:14 2018
From: chbailly at gmail.com (Christophe Bailly)
Date: Sun, 16 Dec 2018 09:21:14 +0100
Subject: [asyncio] Suggestion for a major PEP
Message-ID: <CAPUpzirbxLHPesL5X1ABfzcYH9zYtOh_q8Cg9QHfp47ObJJrUg@mail.gmail.com>

Hello,

I copy paste the main idea from an article I have written:
contextual async
<https://pythonc.home.blog/2018/12/14/premier-article-de-blog/>

"

Imagine you have some code written for monothread. And you want to include
your code in a multithread environment.  Do you need to adapt all your code
which is what you do when you want to migrate to async code ? The answer is
no.

Functionnally these constraints are not justified neither technically

Do we have the tools to do this ? Yes because thanks to boost::context we
can switch context between tasks. When a task suspends, it just calls a
function (the event loop or reactor) to potentially switch to another task.
Just like threads switch contexts?

Async/Await logic has introduced a symetric relation wich introduces
unnecessary contraints. We should just the same logic as thread logic.

"

Read the examples in the article I have developped a prototype in C++ and
everything works perfectly.

My opinion is that sooner or later, it will have to switch to this logic
because chaining async/aswait is a huge contraints and does not make sense
in my opinion.

Maybe I am missing something,

Feel free to give me your feedback.

Regards,


Chris


From mallickaman2003 at gmail.com  Sun Dec 16 04:08:08 2018
From: mallickaman2003 at gmail.com (mallickaman2003 at gmail.com)
Date: Sat, 15 Dec 2018 21:08:08 -1200
Subject: I am facing problem in saving my file.
Message-ID: <5c1615f9.1c69fb81.f6f81.1b09@mx.google.com>



Sent from Mail for Windows 10



From jfong at ms4.hinet.net  Sun Dec 16 04:12:38 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Sun, 16 Dec 2018 01:12:38 -0800 (PST)
Subject: Smarter algo, was Re: 03 digression by brute force
In-Reply-To: <Zs6dncM08ub-a4nBnZ2dnUU78IWdnZ2d@brightview.co.uk>
References: <007d01d491bf$5dd1ab30$19750190$@verizon.net>
 <mailman.32.1544580423.2771.python-list@python.org>
 <fd7993a5-1e04-49b2-819a-2d41d860e0dd@googlegroups.com>
 <puvp23$v6r$1@blaine.gmane.org>
 <012201d4940a$fb490f00$f1db2d00$@verizon.net>
 <mailman.52.1544832804.2771.python-list@python.org>
 <9ddf88ae-6952-46fe-8f07-c76037ffbb7c@googlegroups.com>
 <Zs6dncM08ub-a4nBnZ2dnUU78IWdnZ2d@brightview.co.uk>
Message-ID: <d70b8c1b-924e-4661-8122-17aad24559e0@googlegroups.com>

BlindAnagram at 2018/12/15 UTC+8 PM 8:41:21 wrote:
> On 15/12/2018 09:56, jfong at ms4.hinet.net wrote:
> > Appreciate your thoughtfully analysis on this code. Before generalize it with arbitrary additions, as Peter suggested:-), a recursive version is needed. I may give it a try on this Sunday.
> > 
> > 
> > Avi Gross at 2018/12/15 UTC+8 AM8:13:37 wrote:
> >> REAL SUBJECT: Analysis of alternate algorithms.
> >>
> >> Peter & Jach and anyone interested,
> >>
> >> As Peter said in his altered subject line, Jack changed directions from tweaking an algorithm to trying something quite different.
> >>
> >> Reminder of the problem. 
> >>
> >> Horizontal View:
> >> SEND + MORE = MONEY
> >>
> >> Vertical View:
> >>   SEND
> >> +MORE
> >> ...........
> >> MONEY
> >>
> >> Hard to be precise as I am sticking to plain text in my message. The three words above are meant to be right adjusted.
> >>
> >> When solving it horizontally, Jach and I used variants of a brute force approach. Substitute all possible combinations. He did it in-line and used eval. I did it by making lists of items on both sides and summing the int() of each component and comparing. We tried both our algorithms and his was slower and he profiled that the cause was that eval() was much more expensive as were his use of regular expression functions. For comparison, mine used int() and string manipulation functions and sets and dictionaries.
> >>
> >> But the real puzzle is meant to be solved in a more vertical way by humans using logic. I won't do that here but note we have 4 equations going down but 8 unknowns. And the equations are not unique.
> >>
> >> The rightmost column (I will call it the first as our arithmetic proceeds from right to left) is meant to represent ONES and provides the equation:
> >>
> >> (D+E== Y) or (D+E == Y + 10)
> >>
> >> Worse, for the next column, you either get a "1" carried from the previous addition or not and either pass a "1" along to the next column or not. 4 Possibilities.
> >>
> >> (N+R==E) or (N+R+1==E) or (N+R==E+10) or (N+R+1==E+10)
> >>
> >> Getting a headache yet?
> >>
> >> For a human, they need a way to come up with additional info in terms of constraints.
> >>
> >> There is a guiding inequality that looks like this:
> >>
> >> S is not the same as any of the others. Anytime you solve for another, the list of possible values for S shrinks.
> >> Ditto for each other variable.
> >> Or, since anything plus 0 is itself, then D and E adding up to Y (with no possible carry) cannot be 0.
> >>
> >> But getting a computer to do this might be a bit harder than blunt-force searches. So let's see why Jach's new algorithm was faster.
> >>
> >> The code I am analyzing can be viewed in the archives and will not be entered again:
> >>
> >> https://mail.python.org/pipermail/python-list/2018-December/738454.html
> >>
> >> So what did Jach try in his newer version? It is what I call a vertical approach but one a computer can do easier than a human can or would. I see it is a very specific algorithm that hard codes in these variables as global entities that are altered by a set of nested functions. S, E, N, D, M, O, R, Y. There are approaches that might be better such as passing a dictionary partially filled out from one function to the next as the only one that prints the solution is the final function call.
> >>
> >> So his is not a general solution.
> >>
> >> What interests me as a probable reason this is faster is the number of situations it tries. The earlier versions asked itertools.permutations() to provide all unique combinations of ten tokens in eight positions. So there were 10 choices for the first and 9 for the second and so on adding up to 10!/2! or 1,814,400  different combinations tried. That approaches 2 million.
> >>
> >> Jack broke the problem down into evaluating the units column with a loop like this:
> >>
> >> itertools.permutations(range(10), 3)
> >>
> >> That is 720 possibilities. He then doubled that to 1,440 to consider a carry. Only if the selected values for the three variables in contention (plus a carry) does he go on to call to evaluate the tens column.
> >>
> >> It then shrinks a bit more as he no longer gets the permutations of all 10 digits. He subtracts the three values that might work for the units, so he is asking for permutations of 7 digits, two at a time. That is 42, doubled again to 84 for carry purposes. And this function is not called 1,440 times, but quite a bit fewer. 
> >>
> >> So, similarly, of those 84 loops for tens, he only sometimes calls to evaluate hundreds. As mentioned, the set of 10 digits shrinks some more and this continues upward to functions that evaluate hundreds and thousands  and finally the one evaluating ten thousands pretty much prints out an answer it finds. 
> >>
> >> So overall iterations can be shown to drop. We could add code to measure how many times each function is called and come up with an exact value for this built-in problem. I did and the functions were called this many times:
> >>
> >>>>> counting
> >> {'unit': 1, 'ten': 72, 'hundred': 290, 'thou': 183, '10thou': 196}
> >>>>> sum(counting.values())
> >> 742
> >>
> >> But I also see the permutation function was called 542 times. Most of those runs though were fairly trivial as the combined number of items issued back were only 7,390 as compared to nearly two million in the earlier version. Overall, this is much faster.
> >>
> >> Range() is probably a highly efficient built-in written in C, but it can totally be eliminated in this code as it is a constant. You can write {1,2,2,3,4,5,6,7,8,9} or [0,1] instead of calculating ranges.
> >>
> >> Naturally, this code does not scale up to finding the two solutions for:
> >>
> >> HAWAII + IDAHO +IOWA + OHIO = STATES
> >>
> >> The horizontal versions solved that easily.
> >>
> >> The carry issues get more complex here even if a general solution is written. One approach might be to make a matrix as wide as the widest term and place all entries in it as characters, right justified. You can then work on any number of columns by extracting columns backwards from the right and applying whatever logic, perhaps recursively. The possible carry amounts need to be estimated as no more than about the number of items being added minus one.
> >>
> >> But, I am ready to do other things so I leave it as an exercise for the reader ?
> >>
> >> -----Original Message-----
> >> From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Peter Otten
> >> Sent: Friday, December 14, 2018 3:21 AM
> >> To: python-list at python.org
> >> Subject: Smarter algo, was Re: 03 digression by brute force
> >>
> >> jfong at ms4.hinet.net wrote:
> >>
> >>> Just for fun:-) On my ooold PC, it takes 0.047 seconds to run the 
> >>> following algorithm on the problem 'SNED + MORE == MONEY".
> >>
> >>> def tenThousand(u, Cin):  # Cin == M
> >>>     global n
> >>>     if Cin == M:
> >>>         print(S, E, N, D, '+', M, O, R, E, '==', M, O, N, E, Y)
> >>>         n += 1
> >>
> >> And it probably took under two minutes with the brute force approach.
> >> So if you were only interested in the result the code below were efficient only if it took at most two more minutes to write it;)
> >>
> >> But seriously, the next step is to generalize this to work with arbitrary additions...
> >> More fun!
> 
> There are quite a few Python based solvers for alphametics around on the
> net, for example:
> 
> http://code.activestate.com/recipes/576615-alphametics-solver/
> 
> There is also a long discussion here on ways of doing this:
> 
> https://enigmaticcode.wordpress.com/2016/06/22/solving-alphametics-with-python/

I was shocked when I saw this link! Must take my hat off to its author:-)



From 11akshayghugare at gmail.com  Sun Dec 16 05:43:52 2018
From: 11akshayghugare at gmail.com (Akshay Ghugare)
Date: Sun, 16 Dec 2018 16:13:52 +0530
Subject: installing issue
Message-ID: <CAEYxuWEZkiqHnqMESamOv+P6tYMUAoOVjFW2iQuhsWU6N3bGYg@mail.gmail.com>

 python uninstall problem.jpg
<https://drive.google.com/file/d/1beHYZK0dNZCTUxTs-L43f8PmieVmzKvI/view?usp=drive_web>

python is not working on my pc .after instaling 3.71 version appliction
showing above error .plz fix it as soon as possible..


From prasannakumarbtechit at outlook.com  Sun Dec 16 07:26:56 2018
From: prasannakumarbtechit at outlook.com (Prasanna kumar)
Date: Sun, 16 Dec 2018 12:26:56 +0000
Subject: error
Message-ID: <HK0PR01MB2916DA01D1F131108AD0DA37C9A30@HK0PR01MB2916.apcprd01.prod.exchangelabs.com>


Sir/mam,
Fatal error in launcher: Unable to create process using   this is error which I am not getting clear with it . so, I request you to help me it.


From mytilinaiosvasilis at gmail.com  Sun Dec 16 09:40:23 2018
From: mytilinaiosvasilis at gmail.com (Vasilis Mytilinaios)
Date: Sun, 16 Dec 2018 14:40:23 +0000
Subject: No connection
Message-ID: <CAKgKVtwCMf+LN5OsUKeQpkZC7ea9h=tuPOOwdcv-LoRRAcu8Pg@mail.gmail.com>

Hello,

I'm trying to open the interactive shell but it doesn't let me. It says
that IDLE's subprocess didn't make connection. I unistall and install
python again and still doesn't work.
I was using it properly until 2 days ago.
Any advice on how can I fix it?

Thank you,
Vasilis


From arj.python at gmail.com  Sun Dec 16 14:02:33 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sun, 16 Dec 2018 23:02:33 +0400
Subject: I am facing problem in saving my file.
In-Reply-To: <5c1615f9.1c69fb81.f6f81.1b09@mx.google.com>
References: <5c1615f9.1c69fb81.f6f81.1b09@mx.google.com>
Message-ID: <CADrxXXmQch9dDzOFF0=sqLLR5UzfKTMQ8YRhYh2H5v+3HiaZFg@mail.gmail.com>

error message?

Abdur-Rahmaan Janhangeer
Mauritius


From tjreedy at udel.edu  Sun Dec 16 14:12:45 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 16 Dec 2018 14:12:45 -0500
Subject: No connection
In-Reply-To: <CAKgKVtwCMf+LN5OsUKeQpkZC7ea9h=tuPOOwdcv-LoRRAcu8Pg@mail.gmail.com>
References: <CAKgKVtwCMf+LN5OsUKeQpkZC7ea9h=tuPOOwdcv-LoRRAcu8Pg@mail.gmail.com>
Message-ID: <pv67vc$hfi$1@blaine.gmane.org>

On 12/16/2018 9:40 AM, Vasilis Mytilinaios wrote:
> Hello,
> 
> I'm trying to open the interactive shell but it doesn't let me. It says
> that IDLE's subprocess didn't make connection. 

Read the doc section on possible reasons.
https://docs.python.org/3/library/idle.html#startup-failure


-- 
Terry Jan Reedy



From avigross at verizon.net  Sun Dec 16 19:00:40 2018
From: avigross at verizon.net (Avi Gross)
Date: Sun, 16 Dec 2018 19:00:40 -0500
Subject: Fastest first
Message-ID: <006f01d4959b$8c77adb0$a5670910$@verizon.net>

I have a serious question about what is already available out there in a
herpetologists dream  pythonland.

 

SHORT VERSION: a way to automatically run multiple algorithms in parallel
and kill the rest when one returns an answer.

 

I know I can write this by hand. But it looks like the kind of thing that
would come in handy.

 

MOTIVATION: I recently was discussing alternate GENERAL  ways to solve a
puzzle. Depending on the nature of the input, it seems likely that some
methods will run faster while for other scenarios another might win. It may
not be possible to determine easily which to use so I wonder why not use ALL
of them but stop when an answer arrives.

 

I know how to use all kinds of multitasking in python ranging from threads
running in the same process to multiple external processes that can
communicate with the parent or each other to processes running on different
machines over the internet. You can obviously start algorithm A then B and
as many more as you want in each case. When a (final) solution is reached,
you can send it back to the consolidator (such as a parent) and it can
presumably easily kill local threads by thread number or processes by
process ID. Remote executions might be handled by sending suicide messages
through sockets.

 

Since some algorithms may take orders of magnitude longer than others and
may not ever terminate, letting them continue might be wasteful. But even if
they all share the same CPU, the winner might be slowed down relative to
running alone but perhaps not by much. If they share the same process, the
slowdown for N running together might be I the neighborhood of taking N
times as long. It may be less or more.  If run as another independent
process on a machine which has many other processes running, it may degrade
performance by only a few percent. When running on other machines, no real
impact except the overhead of communications.

 

I am pretty sure there are already many variations out there. When google
searches what has to be a gigantic partially indexed mass of data, I think
it farms out bits of the work to multiple processes and probably processors
as no ONE of them can sift through much of that data in real time. Much of
the search may be done in computer memories. So a search may spawn a massive
attempt and results that come back are consolidated and displayed. Perhaps
after a second or two, no additional data is wanted. It will happily page
the first billion pages it found.

 

What would be nice is to load a module, give it a list of functions to call
and ask to be given an answer with a guarantee that no functions (or
processes)  are still in the running. In real life, this could be complex if
some of your functions invoked a similar method to farm out their work. You
would be killing trees.

 

An example is what happens if you ask for a translation from text written in
an unknown language (I mean Hungarian versus English versus French as an
example.) One solution is to try a list of translators in sequence and if
they return nonsense, try the next. But if you tried them in parallel and
one came back with a 98% probability that it recognized the language because
it found places with a ?zs? combination and found characters like an o with
an umlaut and another o with single/double accents, and regognized a few
words like ?sz?let?snapot? you can pretty much rule out English and French.
You might have a similar approach with multiple variations on SPAM detectors
and call a halt when any one says it seems safe, or vice versa.

 

I would call this a KILL the stragglers survival of the fittest 
 strategy.

 

 

 

 

 

 



From paulbaker8 at gmail.com  Mon Dec 17 05:57:22 2018
From: paulbaker8 at gmail.com (Paul Baker)
Date: Mon, 17 Dec 2018 21:57:22 +1100
Subject: Why do data descriptors (e.g. properties) take priority over instance
 attributes?
Message-ID: <CAAR6NGkwR0ZZFyc2=2=Xox7e+NEutSOKfQ78AitzM3AAz+Jnqw@mail.gmail.com>

When Python looks up an attribute on an object (i.e. when it executes
`o.a`), it uses an interesting priority order [1]. It looks for:

 1. A class attribute that is a data-descriptor (most commonly a property)
 2. An instance attribute
 3. Any other class attribute

We can confirm this using the code below, which creates an object `o`
with an instance attribute `a`, whose class contains a property of the
same name:

    class C:
        def __init__(self):
            self.__dict__['a'] = 1

        @property
        def a(self):
            return 2

    o = C()
    print(o.a)  # Prints 2

Why does Python use this priority order rather than the "naive" order
(instance attributes take priority over all class attributes, as used
by JavaScript)? Python's priority order has a significant drawback: it
makes attribute lookups slower, because instead of just returning an
attribute of `o` if it exists (a common case), Python must first
search `o`'s class *and all its superclasses* for a data-descriptor.

What is the benefit of Python's priority order? It's presumably not
just for the above situation, because having an instance variable and
a property of the same name is very much a corner case (note the need
to use `self.__dict__['a'] = 1` to create the instance attribute,
because the usual `self.a = 1` would invoke the property).

Is there a different situation in which the "naive" lookup order would
cause a problem?

[1] https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/#object-attribute-lookup


From ndbecker2 at gmail.com  Mon Dec 17 07:02:28 2018
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 17 Dec 2018 07:02:28 -0500
Subject: pickle and module versioning
Message-ID: <pv834i$g9u$1@blaine.gmane.org>

I find pickle really handy for saving results from my (simulation) 
experiments.  But recently I realized there is an issue.  Reading the saved 
results requires loading the pickle, which in turn will load any referenced 
modules.  Problem is, what if the modules have changed?

For example, I just re-implemented a python module in C++, in a not quite 
compatible way.  AFAIK, my only choice to not break my setup is to choose a 
different name for the new module.

Has anyone else run into this issue and have any ideas?  I can imagine 
perhaps some kind of module versioning could be used (although haven't 
really thought through the details).

Thanks,
Neal



From cspealma at redhat.com  Mon Dec 17 10:16:04 2018
From: cspealma at redhat.com (Calvin Spealman)
Date: Mon, 17 Dec 2018 10:16:04 -0500
Subject: pickle and module versioning
In-Reply-To: <pv834i$g9u$1@blaine.gmane.org>
References: <pv834i$g9u$1@blaine.gmane.org>
Message-ID: <5c17bdb4.1c69fb81.ab618.b95d@mx.google.com>

This can definitely be a problem, but there are tried and true strategies to deal with it and the best course of action depends a lot on what works for you. Pickle deserialization is just an API instantiating your class instances. That means all the same rules and approaches to API changes apply like any other code changes.

So, if you have to add a field to a class it?ll need to be optional, so previous callers to that API can still use it and the same rule exists here. If you add a new field and its optional or has a default, then un-pickling from an older version should still work if those original fields are still valid.

It does mean the changes you can make are limited, so you need to think through the implications. One approach to mitigating that problem is to isolate the serializing to data classes intended specifically for pickling, so larger changes can happen outside those classes.


From: Neal Becker
Sent: Monday, December 17, 2018 7:02 AM
To: python-list at python.org
Subject: pickle and module versioning

I find pickle really handy for saving results from my (simulation) 
experiments.  But recently I realized there is an issue.  Reading the saved 
results requires loading the pickle, which in turn will load any referenced 
modules.  Problem is, what if the modules have changed?

For example, I just re-implemented a python module in C++, in a not quite 
compatible way.  AFAIK, my only choice to not break my setup is to choose a 
different name for the new module.

Has anyone else run into this issue and have any ideas?  I can imagine 
perhaps some kind of module versioning could be used (although haven't 
really thought through the details).

Thanks,
Neal

-- 
https://mail.python.org/mailman/listinfo/python-list



From __peter__ at web.de  Mon Dec 17 11:40:05 2018
From: __peter__ at web.de (Peter Otten)
Date: Mon, 17 Dec 2018 17:40:05 +0100
Subject: pickle and module versioning
References: <pv834i$g9u$1@blaine.gmane.org>
Message-ID: <pv8jd5$a55$1@blaine.gmane.org>

Neal Becker wrote:

> I find pickle really handy for saving results from my (simulation)
> experiments.  But recently I realized there is an issue.  Reading the
> saved results requires loading the pickle, which in turn will load any
> referenced
> modules.  Problem is, what if the modules have changed?
> 
> For example, I just re-implemented a python module in C++, in a not quite
> compatible way.  AFAIK, my only choice to not break my setup is to choose
> a different name for the new module.
> 
> Has anyone else run into this issue and have any ideas?  I can imagine
> perhaps some kind of module versioning could be used (although haven't
> really thought through the details).

I just tried the renaming part (replacing one.Foo with two.Bar):

$ cat upgrade.py
import pickle
import one

with open("data.dat", "wb") as f:
    pickle.dump(one.Foo(1, 2), f)

with open("data.dat", "rb") as f:
    print(pickle.load(f))

class MyUnpickler(pickle.Unpickler):
    def find_class(self, module, name):
        if (module, name) == ("one", "Foo"):
            module, name = ("two", "Bar")
        return super().find_class(module, name)

with open("data.dat", "rb") as f:
    print(MyUnpickler(f).load())

$ python3.7 upgrade.py
Foo(alpha=1, beta=2)
Bar(alpha=1, beta=2, gamma=42)

In simple cases Bar.__setstate__() should be able to deal with changes in 
object layout. I used

    def __setstate__(self, namespace):
        namespace.setdefault("gamma", 42)
        vars(self).update(namespace)




From massion.francois at gmail.com  Mon Dec 17 13:38:15 2018
From: massion.francois at gmail.com (F Massion)
Date: Mon, 17 Dec 2018 10:38:15 -0800 (PST)
Subject: Variable number of arguments
Message-ID: <f718e553-fa46-463e-bb7f-04b82bba9f7d@googlegroups.com>

My script is used by a web interface which gets some parameters from a form.
The parameters are passed on as arguments like this:

(...)
def start(self, _Inputfile ,_Outputfile ,_Stopwordsfile)

As long as the number of arguments is set (i.e. 3), there are no problems running the script.

Currently I have e.g. 
ARGV[0] = _Inputfile
ARGV[1] = _Outputfile
ARGV[2] = _Stopwordsfile

Now I want to allow a variable number of arguments, i.e. 1..n input files or stopwords lists.

In this case ARGV[0] would become [filename1.txt, filename2.txt,...], but I wonder how ARGV[1] would still remain _Outputfile.

I thought of using the argparse module with nargs='+' for a variable number arguments but it seems to work only with Python 2.7 and I have 3.6 running.

Any suggestions?


From David.Raymond at tomtom.com  Mon Dec 17 13:57:53 2018
From: David.Raymond at tomtom.com (David Raymond)
Date: Mon, 17 Dec 2018 18:57:53 +0000
Subject: Variable number of arguments
In-Reply-To: <f718e553-fa46-463e-bb7f-04b82bba9f7d@googlegroups.com>
References: <f718e553-fa46-463e-bb7f-04b82bba9f7d@googlegroups.com>
Message-ID: <VI1PR07MB57929733060E98560925057587BC0@VI1PR07MB5792.eurprd07.prod.outlook.com>

argparse works fine on 3.x
https://docs.python.org/3.6/library/argparse.html

You can't really have back to back _positional_ arguments with nargs = "+" as it won't be able to tell where you meant one group to end and the next to begin. You'd have to call it with switches like

script.py -if inFile1 inFile2 inFile3 -of outFile -swf stopwordsFile1 stopwordsFile2


-----Original Message-----
From: Python-list [mailto:python-list-bounces+david.raymond=tomtom.com at python.org] On Behalf Of F Massion
Sent: Monday, December 17, 2018 1:38 PM
To: python-list at python.org
Subject: Variable number of arguments

My script is used by a web interface which gets some parameters from a form.
The parameters are passed on as arguments like this:

(...)
def start(self, _Inputfile ,_Outputfile ,_Stopwordsfile)

As long as the number of arguments is set (i.e. 3), there are no problems running the script.

Currently I have e.g. 
ARGV[0] = _Inputfile
ARGV[1] = _Outputfile
ARGV[2] = _Stopwordsfile

Now I want to allow a variable number of arguments, i.e. 1..n input files or stopwords lists.

In this case ARGV[0] would become [filename1.txt, filename2.txt,...], but I wonder how ARGV[1] would still remain _Outputfile.

I thought of using the argparse module with nargs='+' for a variable number arguments but it seems to work only with Python 2.7 and I have 3.6 running.

Any suggestions?
-- 
https://mail.python.org/mailman/listinfo/python-list

From rgaddi at highlandtechnology.invalid  Mon Dec 17 14:30:24 2018
From: rgaddi at highlandtechnology.invalid (Rob Gaddi)
Date: Mon, 17 Dec 2018 11:30:24 -0800
Subject: Python, the stack, and the heap
Message-ID: <pv8tgg$vft$1@dont-email.me>

I've been writing Python for good long while now without thinking too 
hard about this, but I just had a realization this weekend.

Back when the earth's crust was still cooling and we all rode dinosaurs 
to our jobs, local variables got allocated onto the stack, and dynamic 
memory from malloc or DIM explicitly allocated variables on the heap.

Python's objects all have a lifespan dictated by the continued existence 
of references to them and thus can transcend the lifetime of the current 
function in ways not known at translation time.  So am I right in 
thinking that all Python objects are out on the heap?  And that the 
references themselves may or may not wind up on the stack depending on 
what flavor you're running?

Answers to these questions have very little bearing on how I actually 
write Python, mind, but now I'm curious.


-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.


From rosuav at gmail.com  Mon Dec 17 14:42:28 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 18 Dec 2018 06:42:28 +1100
Subject: Python, the stack, and the heap
In-Reply-To: <pv8tgg$vft$1@dont-email.me>
References: <pv8tgg$vft$1@dont-email.me>
Message-ID: <CAPTjJmq8tQVPQ-krjtXsmNTkHRR_7z0ArB6md=x6y+rVriyQUA@mail.gmail.com>

On Tue, Dec 18, 2018 at 6:36 AM Rob Gaddi
<rgaddi at highlandtechnology.invalid> wrote:
>
> I've been writing Python for good long while now without thinking too
> hard about this, but I just had a realization this weekend.
>
> Back when the earth's crust was still cooling and we all rode dinosaurs
> to our jobs, local variables got allocated onto the stack, and dynamic
> memory from malloc or DIM explicitly allocated variables on the heap.
>
> Python's objects all have a lifespan dictated by the continued existence
> of references to them and thus can transcend the lifetime of the current
> function in ways not known at translation time.  So am I right in
> thinking that all Python objects are out on the heap?  And that the
> references themselves may or may not wind up on the stack depending on
> what flavor you're running?
>
> Answers to these questions have very little bearing on how I actually
> write Python, mind, but now I'm curious.

Correct, all Python objects exist on the heap (ignoring the fact that
the language doesn't actually define them in those terms). References
that are from a function's locals could be seen as being on the stack;
in CPython, you can mess around with sys._getframe() to explore the
stack, and each level of the stack has an f_locals attribute that has
all the local name references at that level. But not *all* references
are on the stack; some are attached to modules (those we tend to call
"globals"), or other objects (eg a list's elements), or might even be
core references (eg there's a few references to None, MemoryError, and
such, deep inside the interpreter). But basically all Python objects
are on the heap.

(Except when they're not, but even then, you can pretend that they
are. Optimization is black magic.)

ChrisA


From eryksun at gmail.com  Mon Dec 17 17:10:45 2018
From: eryksun at gmail.com (eryk sun)
Date: Mon, 17 Dec 2018 16:10:45 -0600
Subject: Python, the stack, and the heap
In-Reply-To: <pv8tgg$vft$1@dont-email.me>
References: <pv8tgg$vft$1@dont-email.me>
Message-ID: <CACL+1avH7u=uwF+jcJLFxMOpsV5SyqNLh4Ve7XBMNz1JkeypwA@mail.gmail.com>

On 12/17/18, Rob Gaddi <rgaddi at highlandtechnology.invalid> wrote:
>
> Python's objects all have a lifespan dictated by the continued existence
> of references to them and thus can transcend the lifetime of the current
> function in ways not known at translation time.  So am I right in
> thinking that all Python objects are out on the heap?  And that the
> references themselves may or may not wind up on the stack depending on
> what flavor you're running?

Objects are heap allocated and use reference counting in CPython (i.e.
Python implemented in C). A scope reference in CPython could be a fast
(function) local that's internal to a frame object; a cell object
that's used to share variables in nested scopes; a locals dict item in
an unoptimized scope (e.g. module, class, exec); or a temporary
reference in the frame's stack while evaluating bytecode. Commonly the
latter is from a LOAD_* opcode. For example, here's the implementation
of LOAD_FAST:

        TARGET(LOAD_FAST) {
            PyObject *value = GETLOCAL(oparg);
            if (value == NULL) {
                format_exc_check_arg(PyExc_UnboundLocalError,
                                     UNBOUNDLOCAL_ERROR_MSG,
                                     PyTuple_GetItem(co->co_varnames, oparg));
                goto error;

            Py_INCREF(value);
            PUSH(value);
            FAST_DISPATCH();
        }

The above gets the object pointer from the frame's fastlocals array,
increments the object's reference count, and pushes the pointer on the
frame's stack.

Note that the bytecode evaluation stack is part of the heap-allocated
frame object. It's not the native thread stack (sometimes called the C
stack when code is written in C), but there's a rough correspondence
between frame objects and the native stack, since the interpreter
calls a C function to evaluate a Python frame.


From rosuav at gmail.com  Mon Dec 17 17:45:15 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 18 Dec 2018 09:45:15 +1100
Subject: Python, the stack, and the heap
In-Reply-To: <CACL+1avH7u=uwF+jcJLFxMOpsV5SyqNLh4Ve7XBMNz1JkeypwA@mail.gmail.com>
References: <pv8tgg$vft$1@dont-email.me>
 <CACL+1avH7u=uwF+jcJLFxMOpsV5SyqNLh4Ve7XBMNz1JkeypwA@mail.gmail.com>
Message-ID: <CAPTjJmoTCqw6ke4dPfW=Og9r1WLBbn07z8xEOPAnR=DObNaZ8Q@mail.gmail.com>

On Tue, Dec 18, 2018 at 9:12 AM eryk sun <eryksun at gmail.com> wrote:
> Objects are heap allocated and use reference counting in CPython (i.e.
> Python implemented in C). A scope reference in CPython could be a fast
> (function) local that's internal to a frame object; a cell object
> that's used to share variables in nested scopes; a locals dict item in
> an unoptimized scope (e.g. module, class, exec); or a temporary
> reference in the frame's stack while evaluating bytecode. Commonly the
> latter is from a LOAD_* opcode. For example, here's the implementation
> of LOAD_FAST:

Hang on, you're conflating two different things here. From the
perspective of a Python program, there is a *call stack* which handles
stuff like this:

def f(x): return g(x - 1)
def g(x): return x * 3
print(f(8))

There's two distinct meanings of "x" here, and they're local to their
respective functions. And if you have any form of recursion or
reentrancy, the call stack is essential to keeping the different
locals separate. This is all part of the definition of the Python
language.

The other stack in CPython is the *operand stack*. You can see it
fairly easily with the dis.dis() function.

>>> dis.dis(f)
  1           0 LOAD_GLOBAL              0 (g)
              2 LOAD_FAST                0 (x)
              4 LOAD_CONST               1 (1)
              6 BINARY_SUBTRACT
              8 CALL_FUNCTION            1
             10 RETURN_VALUE

The expression "x - 1" means "fetch x, fetch 1, subtract, leave the
result on the stack". That's a CPython implementation detail, though;
other implementations can do completely different things.

There's no connection between the two stacks. Both of them can refer
to objects, but the operand stack doesn't really come into most
people's thinking, unless they're trying to dig deep into how
refcounting works or some other internal detail of CPython. (Which is
fun - don't get me wrong - but it's not how normal Python programming
is done.)

> Note that the bytecode evaluation stack is part of the heap-allocated
> frame object. It's not the native thread stack (sometimes called the C
> stack when code is written in C), but there's a rough correspondence
> between frame objects and the native stack, since the interpreter
> calls a C function to evaluate a Python frame.

I don't think this is really accurate. The call stack handles Python
functions, the operand stack handles expression evaluation. The
operand stack can have stuff left on it while another function
executes. For instance:

>>> def triangle(x): return x if x < 2 else x + triangle(x - 1)
...
>>> triangle(5)
15
>>> dis.dis(triangle)
  1           0 LOAD_FAST                0 (x)
              2 LOAD_CONST               1 (2)
              4 COMPARE_OP               0 (<)
              6 POP_JUMP_IF_FALSE       12
              8 LOAD_FAST                0 (x)
             10 RETURN_VALUE
        >>   12 LOAD_FAST                0 (x)
             14 LOAD_GLOBAL              0 (triangle)
             16 LOAD_FAST                0 (x)
             18 LOAD_CONST               2 (1)
             20 BINARY_SUBTRACT
             22 CALL_FUNCTION            1
             24 BINARY_ADD
             26 RETURN_VALUE

The recursive branch (from line 12 onwards) fetches up x, then calls
itself recursively, and then adds the return value to what was already
on the stack. The operand stack quite happily carries from one call to
another. (CPython has some protections, I believe, to ensure that the
operand stack doesn't get destroyed by a mismatched load/store or
something, but conceptually, it's just one stack for all functions.)

ChrisA


From brian_christians at hotmail.com  Mon Dec 17 20:20:34 2018
From: brian_christians at hotmail.com (Brian Christiansen)
Date: Mon, 17 Dec 2018 18:20:34 -0700
Subject: polar coordinates?
In-Reply-To: <pujfum$dsb$1@dont-email.me>
References: <pujfum$dsb$1@dont-email.me>
Message-ID: <pv9i14$o5o$1@dont-email.me>

I don't know if this follow up to my original message will even be seen 
because of all the spam advertising e-books that at least from the 
titles appear to have nothing to do with python.

I apologize for being overly wordy in my description, but I suppose I 
was worried that if I did not go into great detail about exactly what I 
was trying to do, someone might suggest a package that solves part of my 
problem, but not all of it.

I downloaded and installed matplotlib.  At first I was a bit worried 
because I have seem to have 3 libraries that contain python "include" 
files: /usr/lib/python/python27, .../python3, and .../python35, and it 
at least appeared to install it into the python3 directory, but my IDE 
uses 3.5.  Well I did some of the tutorials that are on YouTube, and it 
seems to work fine.

Matplotlib makes the very simple plots that I wish to make in just a few 
statements, (scatter plot, etc.), but to make them into representations 
of PI, each dot on the scatter plot or whatever has to be a different 
color: point 1 is color(3), point 2 is color(1), point 3 is color(4), 
etc.  I am not sure if matplotlib is able to make each dot in the 
scatterplot or whatever a differnt color in the single staement that 
makes to plot, or if a loop has to be set up to draw each of the dots 
individually in a different color, but I think it will work and with 
relatively simple code.

Other statements in the thread I will take under advisement, I suppose.
-- 
My Yonkoma: https://www.flickr.com/photos/brian0908/albums/72157680223526176

The E-mail associated with the account is a "spamcatcher" account that I 
got to every couple of months to empty out, and anything sent to it will 
not be seen for probably several months, if it is seen at all.
Brian Christiansen


From rgaddi at highlandtechnology.invalid  Mon Dec 17 20:32:09 2018
From: rgaddi at highlandtechnology.invalid (Rob Gaddi)
Date: Mon, 17 Dec 2018 17:32:09 -0800
Subject: polar coordinates?
In-Reply-To: <pv9i14$o5o$1@dont-email.me>
References: <pujfum$dsb$1@dont-email.me> <pv9i14$o5o$1@dont-email.me>
Message-ID: <pv9imp$qmc$1@dont-email.me>

On 12/17/18 5:20 PM, Brian Christiansen wrote:
> I don't know if this follow up to my original message will even be seen 
> because of all the spam advertising e-books that at least from the 
> titles appear to have nothing to do with python.
> 
> I apologize for being overly wordy in my description, but I suppose I 
> was worried that if I did not go into great detail about exactly what I 
> was trying to do, someone might suggest a package that solves part of my 
> problem, but not all of it.
> 
> I downloaded and installed matplotlib.? At first I was a bit worried 
> because I have seem to have 3 libraries that contain python "include" 
> files: /usr/lib/python/python27, .../python3, and .../python35, and it 
> at least appeared to install it into the python3 directory, but my IDE 
> uses 3.5.? Well I did some of the tutorials that are on YouTube, and it 
> seems to work fine.
> 
> Matplotlib makes the very simple plots that I wish to make in just a few 
> statements, (scatter plot, etc.), but to make them into representations 
> of PI, each dot on the scatter plot or whatever has to be a different 
> color: point 1 is color(3), point 2 is color(1), point 3 is color(4), 
> etc.? I am not sure if matplotlib is able to make each dot in the 
> scatterplot or whatever a differnt color in the single staement that 
> makes to plot, or if a loop has to be set up to draw each of the dots 
> individually in a different color, but I think it will work and with 
> relatively simple code.
> 
> Other statements in the thread I will take under advisement, I suppose.

If I recall correctly you can make the color argument a list with the 
same number of elements as you have points.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.


From rosuav at gmail.com  Mon Dec 17 20:33:55 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 18 Dec 2018 12:33:55 +1100
Subject: polar coordinates?
In-Reply-To: <pv9i14$o5o$1@dont-email.me>
References: <pujfum$dsb$1@dont-email.me> <pv9i14$o5o$1@dont-email.me>
Message-ID: <CAPTjJmqD2UD3a2ACm-PvVWJa6OxfZ5Wk-5oGix-=YHBSuk_ieg@mail.gmail.com>

On Tue, Dec 18, 2018 at 12:26 PM Brian Christiansen
<brian_christians at hotmail.com> wrote:
>
> I don't know if this follow up to my original message will even be seen
> because of all the spam advertising e-books that at least from the
> titles appear to have nothing to do with python.

It's easy enough to see the important information on the mailing list
(python-list at python.org). The ads you mention don't come through to
here.

> I downloaded and installed matplotlib.  At first I was a bit worried
> because I have seem to have 3 libraries that contain python "include"
> files: /usr/lib/python/python27, .../python3, and .../python35, and it
> at least appeared to install it into the python3 directory, but my IDE
> uses 3.5.  Well I did some of the tutorials that are on YouTube, and it
> seems to work fine.

The best way to install Python packages is using pip. Whatever it is
you do to run Python, just add "-m pip" and you then have pip, acting
on the same installation. For instance, I can type "python3 -m pip
install matplotlib", or "python3.7 -m pip install matplotlib" to
install into Python 3.7 (as opposed to my default Python, which is
3.8).

> Matplotlib makes the very simple plots that I wish to make in just a few
> statements, (scatter plot, etc.), but to make them into representations
> of PI, each dot on the scatter plot or whatever has to be a different
> color: point 1 is color(3), point 2 is color(1), point 3 is color(4),
> etc.  I am not sure if matplotlib is able to make each dot in the
> scatterplot or whatever a differnt color in the single staement that
> makes to plot, or if a loop has to be set up to draw each of the dots
> individually in a different color, but I think it will work and with
> relatively simple code.

Not sure, but I think it will be easy enough.

ChrisA


From dieter at handshake.de  Tue Dec 18 03:15:48 2018
From: dieter at handshake.de (dieter)
Date: Tue, 18 Dec 2018 09:15:48 +0100
Subject: pickle and module versioning
References: <pv834i$g9u$1@blaine.gmane.org>
Message-ID: <87imzr6yiz.fsf@handshake.de>

Neal Becker <ndbecker2 at gmail.com> writes:
> ...
> Has anyone else run into this issue

People who use the "ZODB" (= "Zope Object DataBase")
are familiar with it.

> and have any ideas?

There are things, you must not do. In the "ZODB" case, for example,
you must not change whether the class inherits from "persistent.Persistent"
or not.

For other things, there are workarounds:

 * you can use "module" aliases and/or class aliases inside a module
   to handle changes in the package/class structure

 * you can register a "class factory" to provide a "default class"
   for a class not found -- this way, you have at least
   access to the data event though the replacement class
   does not understand it.

 * if a (new) class is found, it must be able to cope
   with the (old) data. You may need to provide
   an appropriate "__setstate__" in the (new) class
   to make this possible.

 * objects created from old data may lack attributes
   which objects created after a class change have.
   You can either add them in "__setstate__" or provide
   default values in the class

 * if you plan drastical changes, migrate your data using
   a different serialization.

   An example is e.g. a switch from "ZODB on Python 2" to
   "ZODB on Python 3". One approach in this case
   is to use "collective.transmogrifier".
   See "https://docs.plone.org/develop/import/index.html#collective-transmogrifier" for details.



From dieter at handshake.de  Tue Dec 18 03:32:52 2018
From: dieter at handshake.de (dieter)
Date: Tue, 18 Dec 2018 09:32:52 +0100
Subject: Variable number of arguments
References: <f718e553-fa46-463e-bb7f-04b82bba9f7d@googlegroups.com>
Message-ID: <87efaf6xqj.fsf@handshake.de>

F Massion <massion.francois at gmail.com> writes:
> My script is used by a web interface which gets some parameters from a form.
> The parameters are passed on as arguments like this:
>
> (...)
> def start(self, _Inputfile ,_Outputfile ,_Stopwordsfile)
>
> As long as the number of arguments is set (i.e. 3), there are no problems running the script.
>
> Currently I have e.g. 
> ARGV[0] = _Inputfile
> ARGV[1] = _Outputfile
> ARGV[2] = _Stopwordsfile
>
> Now I want to allow a variable number of arguments, i.e. 1..n input files or stopwords lists.
>
> In this case ARGV[0] would become [filename1.txt, filename2.txt,...], but I wonder how ARGV[1] would still remain _Outputfile.
>
> I thought of using the argparse module with nargs='+' for a variable number arguments but it seems to work only with Python 2.7 and I have 3.6 running.

"argparse" works with both Python 2.7 and Python 3 (above 3.2, I think).

*BUT* "argparse" is used for parsing command lines and has nothing
to do with function signatures. Thus, "argparse" would
only help, if the "parameters from a form" contain an arguments string
which must be parsed into individual arguments.

For a web interface, it would be much more natural
that the arguments ("input", "output", "stop words") come
from individual form fields. In this case, no parsing would
be necessary.

Depending on what form library your web interface uses,
it may support complex form values (such as lists) directly.
One example is "z3c.form".



From dieter at handshake.de  Tue Dec 18 03:54:48 2018
From: dieter at handshake.de (dieter)
Date: Tue, 18 Dec 2018 09:54:48 +0100
Subject: Why do data descriptors (e.g. properties) take priority over
 instance attributes?
References: <CAAR6NGkwR0ZZFyc2=2=Xox7e+NEutSOKfQ78AitzM3AAz+Jnqw@mail.gmail.com>
Message-ID: <87a7l36wpz.fsf@handshake.de>

Paul Baker <paulbaker8 at gmail.com> writes:
> When Python looks up an attribute on an object (i.e. when it executes
> `o.a`), it uses an interesting priority order [1]. It looks for:
>
>  1. A class attribute that is a data-descriptor (most commonly a property)
>  2. An instance attribute
>  3. Any other class attribute
> ...
> Why does Python use this priority order rather than the "naive" order
> (instance attributes take priority over all class attributes, as used
> by JavaScript)? Python's priority order has a significant drawback: it
> makes attribute lookups slower, because instead of just returning an
> attribute of `o` if it exists (a common case), Python must first
> search `o`'s class *and all its superclasses* for a data-descriptor.
>
> What is the benefit of Python's priority order? It's presumably not
> just for the above situation, because having an instance variable and
> a property of the same name is very much a corner case (note the need
> to use `self.__dict__['a'] = 1` to create the instance attribute,
> because the usual `self.a = 1` would invoke the property).
>
> Is there a different situation in which the "naive" lookup order would
> cause a problem?

"special methods" are looked up in the same way - at least when
called indirectly. There is a section in the "library reference"
explaining why this special lookup is used in this case.
In old Python versions, "special methods" have been looked up in the normal
way; changing the lookup has been a major incompatibility. Thus,
apparently, there have been good reasons for the change.



From banerjee.rohan98 at gmail.com  Tue Dec 18 03:55:45 2018
From: banerjee.rohan98 at gmail.com (banerjee.rohan98 at gmail.com)
Date: Tue, 18 Dec 2018 00:55:45 -0800 (PST)
Subject: ipython does not work with latest version of prompt-toolkit
In-Reply-To: <87wovgz375.fsf@munus.decebal.nl>
References: <87wovgz375.fsf@munus.decebal.nl>
Message-ID: <0b12b430-6c36-42e5-b773-b57890346eaa@googlegroups.com>

On Sunday, 3 June 2018 13:58:14 UTC+5:30, Cecil Westerhof  wrote:
> When executing:
>     pip3 list --no-cache-dir --outdated
> 
> I got:
>     prompt-toolkit 1.0.15    2.0.1    wheel
>     PyGObject      3.28.2    3.28.3   sdist
>     youtube-dl     2018.5.30 2018.6.2 wheel
> 
> So I executed:
>     pip3 install --upgrade prompt-toolkit PyGObject youtube-dl
> 
> This gave:
>     Successfully installed PyGObject-3.28.3 prompt-toolkit-2.0.1 youtube-dl-2018.6.2
>     ipython 6.4.0 has requirement prompt-toolkit<2.0.0,>=1.0.15, but you'll have prompt-toolkit 2.0.1 which is incompatible.
> 
> And when I now execute ipython3, I get:
>     Traceback (most recent call last):
>       File "/usr/local/bin/ipython3", line 7, in <module>
>         from IPython import start_ipython
>       File "/usr/local/lib/python3.5/dist-packages/IPython/__init__.py", line 55, in <module>
>         from .terminal.embed import embed
>       File "/usr/local/lib/python3.5/dist-packages/IPython/terminal/embed.py", line 16, in <module>
>         from IPython.terminal.interactiveshell import TerminalInteractiveShell
>       File "/usr/local/lib/python3.5/dist-packages/IPython/terminal/interactiveshell.py", line 22, in <module>
>         from prompt_toolkit.shortcuts import create_prompt_application, create_eventloop, create_prompt_layout, create_output
>     ImportError: cannot import name 'create_prompt_application'
> 
> When I now execute:
>     pip3 list --no-cache-dir --outdated
> 
> I do not get output. So pip3 thinks everything is OK.
> 
> How do I fix this? Or is the expected that ipython3 will be updated
> shortly?
> 
> -- 
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof

I downgraded to the following versions and I go it working:
Python 3.6.5
jupyter 1.0.0
jupyter-client 5.2.3
jupyter-console 5.2.0
jupyter-core 4.4.0
ipython 6.4.0
prompt-toolkit to the version 1.0.15


From frank at chagford.com  Tue Dec 18 05:50:28 2018
From: frank at chagford.com (Frank Millman)
Date: Tue, 18 Dec 2018 12:50:28 +0200
Subject: Complex 'compare'
Message-ID: <pvaj9k$ib9$1@blaine.gmane.org>

Hi all

I want to compare two tuples. They each represent a row in a database, and 
each element represents a column, so I will use that terminology.

I need to know if one row is greater than or less than the other. The sort 
sequence can be complex - one or more columns, each of which can be sorted 
ascending or descending.

Below is the function I have come up with. Can anyone see any problem with 
it, or suggest a better way to do it?

I have taken one short cut. If all sort columns compare equal, it will 
return True, which is wrong. In practice I ensure that one of the sort 
columns will be the primary key, so the two rows should never be equal.

Thanks

Frank Millman

def compare(source_row, target_row, order, compare_type):

    # source_row - the row I want to compare - some sort columns could 
contain None
    # target_row - the row I want to compare it with - no sort columns will 
contain None
    # order - a list of 2-part tuples defining the sort sequence - the 
column number, and True if descending else False
    # compare_type - either 'gt' or 'lt'

    def compare_col(source_col, target_col, desc, compare_type):
        if compare_type == 'gt':
            if desc:
                return True if source_col is None else source_col < 
target_col
            else:
                return False if source_col is None else source_col > 
target_col
        elif compare_type == 'lt':
            if desc:
                return False if source_col is None else source_col > 
target_col
            else:
                return True if source_col is None else source_col < 
target_col

    for pos, desc in order:
        source_col = source_row[pos]
        target_col = target_row[pos]
        if compare_col(source_col, target_col, desc, compare_type):
            return True
        if source_col != target_col:
            return False
        # if we get here they are equal -
        #    compare the next column in the sort sequence
    return True




From rosuav at gmail.com  Tue Dec 18 06:03:40 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 18 Dec 2018 22:03:40 +1100
Subject: Complex 'compare'
In-Reply-To: <pvaj9k$ib9$1@blaine.gmane.org>
References: <pvaj9k$ib9$1@blaine.gmane.org>
Message-ID: <CAPTjJmpLuyFf04AT+34VraJ5itDvNySVJspEv=DdWDSMMSF88w@mail.gmail.com>

On Tue, Dec 18, 2018 at 9:52 PM Frank Millman <frank at chagford.com> wrote:
> I need to know if one row is greater than or less than the other. The sort
> sequence can be complex - one or more columns, each of which can be sorted
> ascending or descending.
>
> Below is the function I have come up with. Can anyone see any problem with
> it, or suggest a better way to do it?

I'm not sure what the difference is between "compare_type" and "desc".
You have four options here:

* gt, desc
    True if source_col is None else source_col < target_col
* gt, not desc
    False if source_col is None else source_col > target_col
* lt, desc
    False if source_col is None else source_col > target_col
* lt, not desc
    True if source_col is None else source_col < target_col

The way this is currently coded, these come in two perfect pairs. The
"desc" flag is exactly the same as inverting the gt/lt status, or if
you prefer, toggling gt/lt is exactly the same as inverting every
comparison. My best understanding of it is that your rich comparison
function is unable to express the sentiment "they're equal", which
then has to be handled separately; but I don't know what would happen
if you're working with values that aren't totally ordered.

If you can't in any way simplify the overall logic, and you CAN assume
a total ordering (which means that exactly one of "a < b", "a == b",
"a > b" will be true), then what I'd recommend is doing a bit of
algebra. Something like:

for pos, desc in order:
    if compare_type == "lt": desc = not desc
    source_col = source_row[pos]
    target_col = target_row[pos]
    if source_col == target_col: continue
    if source_col is None: return False
    return (source_col < target_col) == desc

I may have the logic wrong, as I'm not sure I fully understand your
code, but this is the kind of way that I'd try to do it.

ChrisA


From frank at chagford.com  Tue Dec 18 06:38:35 2018
From: frank at chagford.com (Frank Millman)
Date: Tue, 18 Dec 2018 13:38:35 +0200
Subject: Complex 'compare'
In-Reply-To: <CAPTjJmpLuyFf04AT+34VraJ5itDvNySVJspEv=DdWDSMMSF88w@mail.gmail.com>
References: <pvaj9k$ib9$1@blaine.gmane.org>
 <CAPTjJmpLuyFf04AT+34VraJ5itDvNySVJspEv=DdWDSMMSF88w@mail.gmail.com>
Message-ID: <pvam3s$dme$1@blaine.gmane.org>

"Chris Angelico"  wrote in message 
news:CAPTjJmpLuyFf04AT+34VraJ5itDvNySVJspEv=DdWDSMMSF88w at mail.gmail.com...
>
> On Tue, Dec 18, 2018 at 9:52 PM Frank Millman <frank at chagford.com> wrote:
> > I need to know if one row is greater than or less than the other. The 
> > sort
> > sequence can be complex - one or more columns, each of which can be 
> > sorted
> > ascending or descending.
> >
> > Below is the function I have come up with. Can anyone see any problem 
> > with
> > it, or suggest a better way to do it?
>
> I'm not sure what the difference is between "compare_type" and "desc".
> You have four options here:
>
> * gt, desc
>     True if source_col is None else source_col < target_col
> * gt, not desc
>     False if source_col is None else source_col > target_col
> * lt, desc
>     False if source_col is None else source_col > target_col
> * lt, not desc
>     True if source_col is None else source_col < target_col
>
> The way this is currently coded, these come in two perfect pairs.
>

Yes, now that you point it out, there is definitely scope for shortening the 
code, as that is effectively duplication.

My main concern was whether my algorithm had any flaws in it. I will keep 
testing to look for corner cases.

I can assume total ordering. The background is that I want to find the 
position of a row in a table according to some sequence. I can't read in the 
entire table, as it could contain millions of rows. So I create a 
server-side cursor in the desired sequence, and then perform a binary search 
on it. So far it is working well.

Thanks

Frank




From h.kheti.serpentcs at gmail.com  Tue Dec 18 07:28:28 2018
From: h.kheti.serpentcs at gmail.com (h.kheti.serpentcs at gmail.com)
Date: Tue, 18 Dec 2018 04:28:28 -0800 (PST)
Subject: How to use multiple instances of the same COM object at the same
 time
In-Reply-To: <kii70i$j5f$1@speranza.aioe.org>
References: <kii70i$j5f$1@speranza.aioe.org>
Message-ID: <fb13a1af-8d5d-4206-842e-d434b856fc2f@googlegroups.com>

On Friday, March 22, 2013 at 11:43:50 PM UTC+5:30, Jan wrote:
> Dear Usenet readers,
> 
> Our company wants to implement about 40 biometric access control devices 
> which have a proprietary DLL with a COM object in it.
> 
> I've done some research about COM objects and how to use them.
> Now, I need to connect to every device separately and register for real 
> time events.  So, I need about 40 instances of the COM object 
> simultaneously connected and registered to catch all events of all devices.
> 
> I assumed I needed threading for this, and I have done a simple 
> implementation.
> 
> All I want to ask is: Does anybody see something that I could have done 
> in a better way? I'm new to COM objects in Python!
> 
> The program runs fine, it is very basic and I tested it with 3 devices 
> connected at the same time.  In production this will be 40!
> 
> Thanks in advance
> 
> Jan
> 
> Source code:
> 
> # Based on eventappartmentthreading from the win32com demo
> 
> import sys, os, time
> import win32com.client
> import win32api
> import win32event
> # sys.coinit_flags not set, so pythoncom initializes apartment-threaded.
> import pythoncom
> from threading import Thread
> 
> threadlist = dict() # this stores the device ids
>                      # using the threadid as the key field
> 
> class ZkEvents:
>      def __init__(self):
>          self.event = win32event.CreateEvent(None, 0, 0, None)
>          thread = win32api.GetCurrentThreadId()
>          print "thread %s " % thread
> 
>          self.dev_id = threadlist[thread]
>          print "Thread Connected For ID %s " % self.dev_id
> 
>      def OnFinger(self):
>          print self.dev_id,
>          print "OnFinger"
> 
>      def OnVerify(self, iUserID=pythoncom.Empty):
>          print self.dev_id,
>          print "OnVerify: %s" % iUserID
> 
>      def OnKeyPress(self, iKey=pythoncom.Empty):
>          print self.dev_id,
>          print "KeyPress: %s" % iKey
> 
> def WaitWhileProcessingMessages(event, timeout = 60):
>      start = time.clock()
>      while True:
>          # Wake 4 times a second - we can't just specify the
>          # full timeout here, as then it would reset for every
>          # message we process.
>          rc = win32event.MsgWaitForMultipleObjects( (event,), 0,
>                                  250,
>                                  win32event.QS_ALLEVENTS)
>          if rc == win32event.WAIT_OBJECT_0:
>              # event signalled - stop now!
>              return True
>          if (time.clock() - start) > timeout:
>              # Timeout expired.
>              return False
>          # must be a message.
>          pythoncom.PumpWaitingMessages()
> 
> def TestZkEvents(ip, dev_id):
>      thread = win32api.GetCurrentThreadId()
>      print 'TestZkEvents created ZK object on thread %d'%thread
>      threadlist[thread] = dev_id
> 
>      pythoncom.CoInitialize()
>      zk = win32com.client.DispatchWithEvents("zkemkeeper.ZKEM", ZkEvents)
> 
>      if zk.Connect_Net(ip, 4370):
>          print "Connect %s OK" % ip
>      else:
>          print "Connect %s Failed" % ip
>      try:
>          if zk.RegEvent(1, 65535): # 65535 = register for all events
>              print "RegEvent %s, %s OK" % (ip, dev_id)
>          else:
>              print "RegEvent %s, %s Failed" % (ip, dev_id)
>      except pythoncom.com_error, details:
>          print "Warning - could not open the test HTML file", details
> 
>      # Wait for the event to be signaled while pumping messages.
>      while True:
>          if not WaitWhileProcessingMessages(zk.event):
>              print "No Events From %s During Last Minute" % dev_id
> 
>      zk = None
> 
> if __name__=='__main__':
>      # this should be a for loop with database fields...
>      t1 = Thread(target=TestZkEvents, args=('192.168.1.211',40))
>      t1.start()
> 
>      t2 = Thread(target=TestZkEvents, args=('192.168.1.212',45))
>      t2.start()
> 
>      t3 = Thread(target=TestZkEvents, args=('192.168.1.213',49))
>      t3.start()




 I Found This Error Plz Help,
  File "C:\Python27\pyt32\lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "C:\Python27\pyt32\lib\threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "C:\Users\Kasim-SCS\Desktop\hk\zkem\v10_biometric_ERPpeek.py", line 266,
in TestZkEvents
    zk = DispatchWithEvents("zkemkeeper.ZKEM", ZkEvents)
  File "C:\Python27\pyt32\lib\site-packages\win32com\client\__init__.py", line 2
53, in DispatchWithEvents
    disp = Dispatch(clsid)
  File "C:\Python27\pyt32\lib\site-packages\win32com\client\__init__.py", line 9
5, in Dispatch
    dispatch, userName = dynamic._GetGoodDispatchAndUserName(dispatch,userName,c
lsctx)
  File "C:\Python27\pyt32\lib\site-packages\win32com\client\dynamic.py", line 11
4, in _GetGoodDispatchAndUserName
    return (_GetGoodDispatch(IDispatch, clsctx), userName)
  File "C:\Python27\pyt32\lib\site-packages\win32com\client\dynamic.py", line 91
, in _GetGoodDispatch
    IDispatch = pythoncom.CoCreateInstance(IDispatch, None, clsctx, pythoncom.II
D_IDispatch)
com_error: (-2147221005, 'Invalid class string', None, None)


How To Solve DispatchWithEnvent Error ??



From avigross at verizon.net  Tue Dec 18 09:30:09 2018
From: avigross at verizon.net (Avi Gross)
Date: Tue, 18 Dec 2018 09:30:09 -0500
Subject: Complex 'compare'
In-Reply-To: <pvaj9k$ib9$1@blaine.gmane.org>
References: <pvaj9k$ib9$1@blaine.gmane.org>
Message-ID: <007a01d496de$2dbf60d0$893e2270$@verizon.net>

Frank,

I am not commenting on your specific code, just asking a question.

If you have two tuples of the same size it does seem that python evaluates
them in the order you want just by doing something like this example I made
with a 2-tuple:

>>> (1, 2) > (2,1)
False
>>> (1, 2) > (1,1)
True
>>> (1, 1) > (1,1)
False

So if you take your database entries and re-order them so the keys you want
to sort by are in order from left to right, and you don't include any that
you don't want to sort by, the above technique may work well enough. Of
course, you may want to re-establish the order and contents afterward.

Of course if you have some None, that messes things up:

>>> (1, 1) > (1,None)
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    (1, 1) > (1,None)
TypeError: '>' not supported between instances of 'int' and 'NoneType'

Unless, of course, they are both None.

>>> (1, None) > (1,None)
False

For python 2.x there was a cmp() function that might have met your needs. It
was removed in 3.x 

https://docs.python.org/3.0/whatsnew/3.0.html

" The cmp() function should be treated as gone, and the __cmp__() special
method is no longer supported. Use __lt__() for sorting, __eq__() with
__hash__(), and other rich comparisons as needed. (If you really need the
cmp() functionality, you could use the expression (a > b) - (a < b) as the
equivalent for cmp(a, b).)"

But as noted this does not work with your None. 

Final comment meant as humor. The following will not work even though it is
complex!

>>> (1 + 2j, 1 - 2j) > (1 + 3j,2 +4j)
Traceback (most recent call last):
  File "<pyshell#11>", line 1, in <module>
    (1 + 2j, 1 - 2j) > (1 + 3j,2 +4j)
TypeError: '>' not supported between instances of 'complex' and 'complex'

And the reason is obvious as complex numbers have two dimensions so you
might need to compare something like their magnitude as in the length of a
vector ...

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Frank Millman
Sent: Tuesday, December 18, 2018 5:50 AM
To: python-list at python.org
Subject: Complex 'compare'

Hi all

I want to compare two tuples. They each represent a row in a database, and
each element represents a column, so I will use that terminology.

I need to know if one row is greater than or less than the other. The sort
sequence can be complex - one or more columns, each of which can be sorted
ascending or descending.

Below is the function I have come up with. Can anyone see any problem with
it, or suggest a better way to do it?

I have taken one short cut. If all sort columns compare equal, it will
return True, which is wrong. In practice I ensure that one of the sort
columns will be the primary key, so the two rows should never be equal.

Thanks

Frank Millman

def compare(source_row, target_row, order, compare_type):

    # source_row - the row I want to compare - some sort columns could
contain None
    # target_row - the row I want to compare it with - no sort columns will
contain None
    # order - a list of 2-part tuples defining the sort sequence - the
column number, and True if descending else False
    # compare_type - either 'gt' or 'lt'

    def compare_col(source_col, target_col, desc, compare_type):
        if compare_type == 'gt':
            if desc:
                return True if source_col is None else source_col <
target_col
            else:
                return False if source_col is None else source_col >
target_col
        elif compare_type == 'lt':
            if desc:
                return False if source_col is None else source_col >
target_col
            else:
                return True if source_col is None else source_col <
target_col

    for pos, desc in order:
        source_col = source_row[pos]
        target_col = target_row[pos]
        if compare_col(source_col, target_col, desc, compare_type):
            return True
        if source_col != target_col:
            return False
        # if we get here they are equal -
        #    compare the next column in the sort sequence
    return True


--
https://mail.python.org/mailman/listinfo/python-list



From ian.g.kelly at gmail.com  Tue Dec 18 09:54:06 2018
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Tue, 18 Dec 2018 07:54:06 -0700
Subject: Why do data descriptors (e.g. properties) take priority over
 instance attributes?
In-Reply-To: <CAAR6NGkwR0ZZFyc2=2=Xox7e+NEutSOKfQ78AitzM3AAz+Jnqw@mail.gmail.com>
References: <CAAR6NGkwR0ZZFyc2=2=Xox7e+NEutSOKfQ78AitzM3AAz+Jnqw@mail.gmail.com>
Message-ID: <CALwzidkC21+9Fr++qCcQfFY4Zt8p8H7_CnON7C01xnPDgcQn_Q@mail.gmail.com>

On Mon, Dec 17, 2018, 12:09 PM Paul Baker <paulbaker8 at gmail.com wrote:

> When Python looks up an attribute on an object (i.e. when it executes
> `o.a`), it uses an interesting priority order [1]. It looks for:
>
>  1. A class attribute that is a data-descriptor (most commonly a property)
>  2. An instance attribute
>  3. Any other class attribute
>
> We can confirm this using the code below, which creates an object `o`
> with an instance attribute `a`, whose class contains a property of the
> same name:
>
>     class C:
>         def __init__(self):
>             self.__dict__['a'] = 1
>
>         @property
>         def a(self):
>             return 2
>
>     o = C()
>     print(o.a)  # Prints 2
>
> Why does Python use this priority order rather than the "naive" order
> (instance attributes take priority over all class attributes, as used
> by JavaScript)? Python's priority order has a significant drawback: it
> makes attribute lookups slower, because instead of just returning an
> attribute of `o` if it exists (a common case), Python must first
> search `o`'s class *and all its superclasses* for a data-descriptor.
>
> What is the benefit of Python's priority order? It's presumably not
> just for the above situation, because having an instance variable and
> a property of the same name is very much a corner case (note the need
> to use `self.__dict__['a'] = 1` to create the instance attribute,
> because the usual `self.a = 1` would invoke the property).
>
> Is there a different situation in which the "naive" lookup order would
> cause a problem?
>

It would create a disparity between lookup and assignment. When you assign
to an attribute that has a data descriptor, it always invokes the
descriptor. If it preferred to assign to the instance dict instead, the
descriptor would never be invited.

Now, suppose you have an object with a data descriptor, and an instance
attribute of 42. We read the attribute and get 42 back, which is not what
we want, so we assign 111 to it, then we read it back again, and the value
is still 42! What gives? We're not able to overwrite or delete the value
that were reading, and it's because an unexpected value crept into the
instance dict somehow. The storage of a data descriptor value is expected
to be defined by the descriptor, not by using the same key in the instance
dict (although this scheme also means that the data descriptor could use
that dict entry for its own storage if it wants).

The reason you can do this for a non-data descriptor is because with no set
or delete, they don't define their own storage, and you can thus set the
instance attribute if you want to override the descriptor.

>


From avigross at verizon.net  Tue Dec 18 09:59:51 2018
From: avigross at verizon.net (Avi Gross)
Date: Tue, 18 Dec 2018 09:59:51 -0500
Subject: Complex 'compare'
In-Reply-To: <pvam3s$dme$1@blaine.gmane.org>
References: <pvaj9k$ib9$1@blaine.gmane.org>
 <CAPTjJmpLuyFf04AT+34VraJ5itDvNySVJspEv=DdWDSMMSF88w@mail.gmail.com>
 <pvam3s$dme$1@blaine.gmane.org>
Message-ID: <008901d496e2$5441a2f0$fcc4e8d0$@verizon.net>

Frank,

Thanks for explaining.

It looks like you might want a shortcut compare done in an iterative or
recursive way that returns as soon as a discrepancy happens in the direction
you are comparing. If they are equal, you fall off the end.

A pythonic way is to use a loop as in:

>>> flag = True
>>> for alpha, beta in zip(first, second):
	if (alpha < beta):
		flag = False
		break

The above is not the full code, just an example. In that loop you would need
to check for None or anything else. You need to determine if the match was
found or if it is less than versus greater than so you can continue your
binary search.

There are many tricks (ahem, techniques) people use. Some compare functions
that need to return a trinary answer will return 1 for Greater than, -1 for
Less than and 0 for equal. The function calling it can use techniques to
branch based on that such as using the return code to index a dictionary
containing keys -1,0, and 1 and a value that is the function to call to
fetch the next item above or below. 

Another method could be to have the function return a tuple of results. If
you return (found, comparison) as in:

(found, comparison) = function(...)

Then if the two are equal, return True in the first variable and if it is
False, examine the second variable to see if it is True or False.

And, of course, you can just do two comparisons in the first place. One for
"==" to find if you found it and the other for a test to see if you need to
search before or after the current cursor.

The quality test might work well in a try statement. Something like this
works fine:

>>> (1,1, None) == (1,1,None)
True

And it seems resistant to many errors because it is testing equality and
objects of different types are often by definition not equal.

>>> (1,1, None) == (1,1,None)
True
>>> (1,1, 2) == (1,1,None)
False
>>> (1,1, "two") == (1,1,2)
False
>>> (1,2) == (1,2,3)
False

If you want to make sure some other errors do not creep in, place it in a
try/catch statement as any error would also be a false.

Just some thoughts.




-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Frank Millman
Sent: Tuesday, December 18, 2018 6:39 AM
To: python-list at python.org
Subject: Re: Complex 'compare'

"Chris Angelico"  wrote in message
news:CAPTjJmpLuyFf04AT+34VraJ5itDvNySVJspEv=DdWDSMMSF88w at mail.gmail.com...
>
> On Tue, Dec 18, 2018 at 9:52 PM Frank Millman <frank at chagford.com> wrote:
> > I need to know if one row is greater than or less than the other. 
> > The sort sequence can be complex - one or more columns, each of 
> > which can be sorted ascending or descending.
> >
> > Below is the function I have come up with. Can anyone see any 
> > problem with it, or suggest a better way to do it?
>
> I'm not sure what the difference is between "compare_type" and "desc".
> You have four options here:
>
> * gt, desc
>     True if source_col is None else source_col < target_col
> * gt, not desc
>     False if source_col is None else source_col > target_col
> * lt, desc
>     False if source_col is None else source_col > target_col
> * lt, not desc
>     True if source_col is None else source_col < target_col
>
> The way this is currently coded, these come in two perfect pairs.
>

Yes, now that you point it out, there is definitely scope for shortening the
code, as that is effectively duplication.

My main concern was whether my algorithm had any flaws in it. I will keep
testing to look for corner cases.

I can assume total ordering. The background is that I want to find the
position of a row in a table according to some sequence. I can't read in the
entire table, as it could contain millions of rows. So I create a
server-side cursor in the desired sequence, and then perform a binary search
on it. So far it is working well.

Thanks

Frank


--
https://mail.python.org/mailman/listinfo/python-list



From eryksun at gmail.com  Tue Dec 18 16:57:11 2018
From: eryksun at gmail.com (eryk sun)
Date: Tue, 18 Dec 2018 15:57:11 -0600
Subject: Python, the stack, and the heap
In-Reply-To: <CAPTjJmoTCqw6ke4dPfW=Og9r1WLBbn07z8xEOPAnR=DObNaZ8Q@mail.gmail.com>
References: <pv8tgg$vft$1@dont-email.me>
 <CACL+1avH7u=uwF+jcJLFxMOpsV5SyqNLh4Ve7XBMNz1JkeypwA@mail.gmail.com>
 <CAPTjJmoTCqw6ke4dPfW=Og9r1WLBbn07z8xEOPAnR=DObNaZ8Q@mail.gmail.com>
Message-ID: <CACL+1auM0vh2kWHUfwXsq4Y6abND3FGADb21AfB243sMjsALRA@mail.gmail.com>

On 12/17/18, Chris Angelico <rosuav at gmail.com> wrote:
>
> Hang on, you're conflating two different things here.

You're probably right that I misinterpreted what the OP meant by
references winding up "on the stack". It's more common for a Python
developer to think of the call stack instead of the implementation
details of local variables and stacks inside frame objects.

>>>> dis.dis(triangle)
>   1           0 LOAD_FAST                0 (x)
>               2 LOAD_CONST               1 (2)
>               4 COMPARE_OP               0 (<)
>               6 POP_JUMP_IF_FALSE       12
>               8 LOAD_FAST                0 (x)
>              10 RETURN_VALUE
>         >>   12 LOAD_FAST                0 (x)
>              14 LOAD_GLOBAL              0 (triangle)
>              16 LOAD_FAST                0 (x)
>              18 LOAD_CONST               2 (1)
>              20 BINARY_SUBTRACT
>              22 CALL_FUNCTION            1
>              24 BINARY_ADD
>              26 RETURN_VALUE
>
> The recursive branch (from line 12 onwards) fetches up x, then calls
> itself recursively, and then adds the return value to what was already
> on the stack. The operand stack quite happily carries from one call to
> another. (CPython has some protections, I believe, to ensure that the
> operand stack doesn't get destroyed by a mismatched load/store or
> something, but conceptually, it's just one stack for all functions.)

CALL_FUNCTION (implemented via call_function in Python/ceval.c)
allocates and evaluates a new frame object when calling a pure Python
function. Inside each CPython frame object is the f_localsplus memory
block, which is divided between local variables and the evaluation
stack. The combination of linked frame objects and their internal
f_localsplus memory blocks is analogous to frames on the native call
stack.

In CPython, the frame-object stack and the native stack are coupled
and grow together, which is why the default recursion limit is
relatively conservative (1000). (Stackless Python severs this
connection.)

Here's what evaluating triangle(2) looks like in terms of the native
call stack in Windows.

Function definition:

    def triangle(x):
        if x < 2:
            return x
        return x + triangle(x - 1)

Statement executed in the REPL:

    >>> triangle(2)

Native call stack (listed from most recent call, in reverse):

    0:000> kc
    Call Site

triangle(1) evaluation:
    python37_d!_PyEval_EvalFrameDefault
    python37_d!function_code_fastcall
    python37_d!_PyFunction_FastCallKeywords
    python37_d!call_function

triangle(2) evaluation:
    python37_d!_PyEval_EvalFrameDefault
    python37_d!function_code_fastcall
    python37_d!_PyFunction_FastCallKeywords
    python37_d!call_function

REPL evaluation:
    python37_d!_PyEval_EvalFrameDefault
    python37_d!_PyEval_EvalCodeWithName
    python37_d!PyEval_EvalCodeEx
    python37_d!PyEval_EvalCode
    python37_d!run_mod
    python37_d!PyRun_InteractiveOneObjectEx
    python37_d!PyRun_InteractiveLoopFlags

Python interpreter startup:
    python37_d!PyRun_AnyFileExFlags
    python37_d!pymain_run_file
    python37_d!pymain_run_filename
    python37_d!pymain_run_python
    python37_d!pymain_main
    python37_d!Py_Main

C runtime startup:
    python_d!wmain
    python_d!invoke_main
    python_d!__scrt_common_main_seh
    python_d!__scrt_common_main
    python_d!wmainCRTStartup

Windows thread startup:
    KERNEL32!BaseThreadInitThunk

NT thread startup:
    ntdll!RtlUserThreadStart


From brian_christians at hotmail.com  Tue Dec 18 20:01:38 2018
From: brian_christians at hotmail.com (Brian Christiansen)
Date: Tue, 18 Dec 2018 18:01:38 -0700
Subject: polar coordinates?
In-Reply-To: <pujfum$dsb$1@dont-email.me>
References: <pujfum$dsb$1@dont-email.me>
Message-ID: <pvc59k$ics$1@dont-email.me>

I don't think what I am currently doing is heat maps, but at least from 
what I have read, they could be adapted to "visualiztions of PI."

Toward that end I decided to put a link to the video that inspired me, a 
link to where I got the graphics.py package (or at least where I think I 
got it), and the actual program I currently have.

Numberphile video about visualiztion of PI: 
https://www.youtube.com/watch?v=NPoj8lk9Fo4

Where I got graphics.py, the graphics package I am currently using (this 
is a link to the actual file I am using: 
tmcsp.wartburg.edu/zelle/pyhon/graphics.py

The program I currently have working.  It is a bit simplistic, it does 
not vary the size of the grid that is made, etc.  It also looks like to 
me that I did not really update my comments that described how big I was 
making my dots to match how big I was actually making my dots.  Also, 
simply cutting and pasting it, even if you have graphics.py in the 
"python path" may not work, since sometimes usenet seems to mess with 
spaces.:

from graphics import *
import random

pi1000 = 
[3,1,4,1,5,9,2,6,5,3,5,8,9,7,9,3,2,3,8,4,6,2,6,4,3,3,8,3,2,7,9,5,0,2,8,
           8,4,1,9,7,1,6,9,3,9,9,3,7,5,1,0, 
5,8,2,0,9,7,4,9,4,4,5,9,2,3,0,7,8,1,
 
6,4,0,6,2,8,6,2,0,8,9,9,8,6,2,8,0,3,4,8,2,5,3,4,2,1,1,7,0,6,7,9,8,2,1,
 
4,8,0,8,6,5,1,3,2,8,2,3,0,6,6,4,7,0,9,3,8,4,4,6,0,9,5,5,0,5,8,2,2,3,1,
 
7,2,5,3,5,9,4,0,8,1,2,8,4,8,1,1,1,7,4,5,0,2,8,4,1,0,2,7,0,1,9,3,8,5,2,
 
1,1,0,5,5,5,9,6,4,4,6,2,2,9,4,8,9,5,4,9,3,0,3,8,1,9,6,4,4,2,8,8,1,0,9,
 
7,5,6,6,5,9,3,3,4,4,6,1,2,8,4,7,5,6,4,8,2,3,3,7,8,6,7,8,3,1,6,5,2,7,1,
 
2,0,1,9,0,9,1,4,5,6,4,8,5,6,6,9,2,3,4,6,0,3,4,8,6,1,0,4,5,4,3,2,6,6,4,
 
8,2,1,3,3,9,3,6,0,7,2,6,0,2,4,9,1,4,1,2,7,3,7,2,4,5,8,7,0,0,6,6,0,6,3,
 
1,5,5,8,8,1,7,4,8,8,1,5,2,0,9,2,0,9,6,2,8,2,9,2,5,4,0,9,1,7,1,5,3,6,4,
 
3,6,7,8,9,2,5,9,0,3,6,0,0,1,1,3,3,0,5,3,0,5,4,8,8,2,0,4,6,6,5,2,1,3,8,
 
4,1,4,6,9,5,1,9,4,1,5,1,1,6,0,9,4,3,3,0,5,7,2,7,0,3,6,5,7,5,9,5,9,1,9,
 
5,3,0,9,2,1,8,6,1,1,7,3,8,1,9,3,2,6,1,1,7,9,3,1,0,5,1,1,8,5,4,8,0,7,4,
 
4,6,2,3,7,9,9,6,2,7,4,9,5,6,7,3,5,1,8,8,5,7,5,2,7,2,4,8,9,1,2,2,7,9,3,
 
8,1,8,3,0,1,1,9,4,9,1,2,9,8,3,3,6,7,3,3,6,2,4,4,0,6,5,6,6,4,3,0,8,6,0,
 
2,1,3,9,4,9,4,6,3,9,5,2,2,4,7,3,7,1,9,0,7,0,2,1,7,9,8,6,0,9,4,3,7,0,2,
 
7,7,0,5,3,9,2,1,7,1,7,6,2,9,3,1,7,6,7,5,2,3,8,4,6,7,4,8,1,8,4,6,7,6,6,
 
9,4,0,5,1,3,2,0,0,0,5,6,8,1,2,7,1,4,5,2,6,3,5,6,0,8,2,7,7,8,5,7,7,1,3,
 
4,2,7,5,7,7,8,9,6,0,9,1,7,3,6,3,7,1,7,8,7,2,1,4,6,8,4,4,0,9,0,1,2,2,4,
 
9,5,3,4,3,0,1,4,6,5,4,9,5,8,5,3,7,1,0,5,0,7,9,2,2,7,9,6,8,9,2,5,8,9,2,
 
3,5,4,2,0,1,9,9,5,6,1,1,2,1,2,9,0,2,1,9,6,0,8,6,4,0,3,4,4,1,8,1,5,9,8,
 
1,3,6,2,9,7,7,4,7,7,1,3,0,9,9,6,0,5,1,8,7,0,7,2,1,1,3,4,9,9,9,9,9,9,8,
 
3,7,2,9,7,8,0,4,9,9,5,1,0,5,9,7,3,1,7,3,2,8,1,6,0,9,6,3,1,8,5,9,5,0,2,
 
4,4,5,9,4,5,5,3,4,6,9,0,8,3,0,2,6,4,2,5,2,2,3,0,8,2,5,3,3,4,4,6,8,5,0,
 
3,5,2,6,1,9,3,1,1,8,8,1,7,1,0,1,0,0,0,3,1,3,7,8,3,8,7,5,2,8,8,6,5,8,7,
 
5,3,3,2,0,8,3,8,1,4,2,0,6,1,7,1,7,7,6,6,9,1,4,7,3,0,3,5,9,8,2,5,3,4,9,
 
0,4,2,8,7,5,5,4,6,8,7,3,1,1,5,9,5,6,2,8,6,3,8,8,2,3,5,3,7,8,7,5,9,3,7,
 
5,1,9,5,7,7,8,1,8,5,7,7,8,0,5,3,2,1,7,1,2,2,6,8,0,6,6,1,3,0,0,1,9,2,7,
           8,7,6,6,1,1,1,9,5,9,0,9,2,1,6,4,2,0,1,9,8,9]
colors =  ["purple", "blue", "brown","coral","cyan","gray","green","yellow",
            "red","orange"]




def pi_as_dots():
     #make window that is 700 by 700
     win = GraphWin("Visualization of PI",700,700)
     win.setBackground("black")
     for y in range(0,14):
         for x in range (0,14):
             # make a circle that is centered at 14x+7, and 14y+7
             pt = Point((50*x+25),(50*y+25))
             cir = Circle(pt,20)
             cir.setFill(colors[pi1000[y*14+x]])
             cir.draw(win)
             #make a circle that is a little smaller, in the same point, 
with the next color
             cir = Circle(pt,15)
             cir.setFill(colors[pi1000[y*14+x+1]])
             cir.draw(win)


def main():

     #make the simple menuing system

     print("choose how to visualize pi")
     print("d = dots")
     choice = input('--> ')
     if choice == "d":
         pi_as_dots()


main()
-- 
My Yonkoma: https://www.flickr.com/photos/brian0908/albums/72157680223526176

The E-mail associated with the account is a "spamcatcher" account that I 
got to every couple of months to empty out, and anything sent to it will 
not be seen for probably several months, if it is seen at all.
Brian Christiansen


From ant at anthive.com  Tue Dec 18 21:48:55 2018
From: ant at anthive.com (ant)
Date: Tue, 18 Dec 2018 21:48:55 -0500
Subject: setup.py, how to find main program/module?
Message-ID: <n5ooef-a7a.ln1@anthive.com>

  hello,

  i'm getting there...

  yet not quite all the way yet.  i run the 
commands to make the sdist and wheels and upload
it using twine to my test account, but when
i install it and try to run it the ngfp.py is
not found (i've also tried nfgp and main).


test upload:

  https://test.pypi.org/project/ngfp/


  i suspect i've not really understood entry 
points and may not have it specified correctly in
setup.py.

  any help appreciated.  :)


  to install it you'll need to use (to pull in pyglet):

$ pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple ngfp==0.1.5.post3

  thank you and cheers,


  ant


From bintacomputers at gmail.com  Wed Dec 19 00:35:50 2018
From: bintacomputers at gmail.com (Umar Yusuf)
Date: Tue, 18 Dec 2018 21:35:50 -0800 (PST)
Subject: Mask two images with python
Message-ID: <c835f128-fa2c-43be-8be9-a3b3baccc749@googlegroups.com>

Hello there,
How do I supper impose an image design on a transparent png image?

I have tried to use OpenCV's "cv2.bitwise_and" function to no success. I posted the detail question here: https://stackoverflow.com/questions/53791510/python-opencv-mask-and-glow

Thank you for your time.


From dieter at handshake.de  Wed Dec 19 01:57:13 2018
From: dieter at handshake.de (dieter)
Date: Wed, 19 Dec 2018 07:57:13 +0100
Subject: setup.py, how to find main program/module?
References: <n5ooef-a7a.ln1@anthive.com>
Message-ID: <87a7l2gg1i.fsf@handshake.de>

ant <ant at anthive.com> writes:
> ...
>   yet not quite all the way yet.  i run the 
> commands to make the sdist and wheels and upload
> it using twine to my test account, but when
> i install it and try to run it the ngfp.py is
> not found (i've also tried nfgp and main).
> ...
>   i suspect i've not really understood entry 
> points and may not have it specified correctly in
> setup.py.
>
>   any help appreciated.  :)

I think you are right: the problem is likely with the
"entry_points" specification.


My "dm.zopepatches.ztest" package has
      entry_points = dict(
         console_scripts = [
           'ztest = dm.zopepatches.ztest:main',
           ]
        ),
This is responsible for the creation of a script called
"ztest" in Python's "bin" directory.


Does your "entry_points" specification look similar?
Do you find corresponding scripts in Python's "bin" directory?




From marko at pacujo.net  Wed Dec 19 06:44:32 2018
From: marko at pacujo.net (Marko Rauhamaa)
Date: Wed, 19 Dec 2018 13:44:32 +0200
Subject: with exceptions?
References: <exceptions-20181219092909@ram.dialup.fu-berlin.de>
Message-ID: <87h8f9aggv.fsf@elektro.pacujo.net>

ram at zedat.fu-berlin.de (Stefan Ram):

> try:
>     with open( 'file', 'r' ) as f:
>         use( f )
> except Exception as inst:
>     print( inst )
>
>   Is target code the correct way to use ?with? with together
>   with ?except??
>
>   Or is it recommended to continue to use ?finally? in such
>   cases?

I'd advice against such ambiguous use of exceptions. I'd write:

  try:
      f = open('file')
  except WhatEverException as e:
      print(e)
  else:
      with f:
          try:
              use(f)
          except OtherException as e:
              print(e)


Marko


From chris at withers.org  Wed Dec 19 08:15:38 2018
From: chris at withers.org (Chris Withers)
Date: Wed, 19 Dec 2018 13:15:38 +0000
Subject: testfixtures 6.4.0 released - important bug fix! (again)
Message-ID: <f24ba87d-e4b3-5d63-2bea-70ea575d0620@withers.org>

   Hi All,

   I'm afraid I found quite a [1]serious problem with unittest.mock.call
   objects that is important for users of testfixtures' MockPopen helper.

   This release includes a monkey patch to fix that issue as well as a facade
   over both unittest.mock and the rolling backport on pypi, giving
   preference to the latter, so you don't have to change your code if you
   need to flip to the rolling backport for a while when waiting to a bug fix
   to land in Python core.

   This release also include:

     * Add official support for Python 3.7.
     * Drop official support for Python 3.5.
     * Better feedback when [2]call() comparisons fail when using compare().
     * A re-working of  MockPopen to enable it  to handle multiple  processes
       being active at the same time.
     * Fixes to Mocking dates and times documentation.

   The package is on PyPI and a full list of all the links to docs, issue
   trackers and the like can be found here:

   [3]https://github.com/Simplistix/testfixtures

   Any questions, please do ask on the Testing in Python list or on the
   Simplistix open source mailing list...

   cheers,

   Chris

References

   Visible links
   1. https://bugs.python.org/issue35226
   2. (in Python v3.7)
	https://docs.python.org/3/library/unittest.mock.html#unittest.mock.call
   3. https://github.com/Simplistix/testfixtures


From ant at anthive.com  Wed Dec 19 08:52:04 2018
From: ant at anthive.com (ant)
Date: Wed, 19 Dec 2018 08:52:04 -0500
Subject: setup.py, how to find main program/module?
References: <n5ooef-a7a.ln1@anthive.com> <87a7l2gg1i.fsf@handshake.de>
 <mailman.25.1545202642.31622.python-list@python.org>
Message-ID: <41vpef-2n2.ln1@anthive.com>

dieter wrote:
> ant <ant at anthive.com> writes:
>> ...
>>   yet not quite all the way yet.  i run the 
>> commands to make the sdist and wheels and upload
>> it using twine to my test account, but when
>> i install it and try to run it the ngfp.py is
>> not found (i've also tried nfgp and main).
>> ...
>>   i suspect i've not really understood entry 
>> points and may not have it specified correctly in
>> setup.py.
>>
>>   any help appreciated.  :)
>
> I think you are right: the problem is likely with the
> "entry_points" specification.
>
>
> My "dm.zopepatches.ztest" package has
>       entry_points = dict(
>          console_scripts = [
>            'ztest = dm.zopepatches.ztest:main',
>            ]
>         ),
> This is responsible for the creation of a script called
> "ztest" in Python's "bin" directory.
>
>
> Does your "entry_points" specification look similar?

  yes.


> Do you find corresponding scripts in Python's "bin" directory?

  thank you,

  yes, it is there:


=====
(env) me at ant(9)~/src/env/bin$ more ngfp 
#!/home/me/src/env/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'ngfp','console_scripts','ngfp'
__requires__ = 'ngfp'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('ngfp', 'console_scripts', 'ngfp')()
    )


=====

(env) me at ant(11)~/src/test$ ngfp
Traceback (most recent call last):
  File "/home/me/src/env/bin/ngfp", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3126, in <module>
    @_call_aside
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3110, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3139, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 581, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 898, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 784, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'ngfp' distribution was not found and is required by the application

=====

  so, actually, the problem isn't what i expected it to be.
but i'm still not sure how to fix it.  :)


  ant


From ant at anthive.com  Wed Dec 19 09:25:45 2018
From: ant at anthive.com (ant)
Date: Wed, 19 Dec 2018 09:25:45 -0500
Subject: ah, progress...
References: <n5ooef-a7a.ln1@anthive.com> <87a7l2gg1i.fsf@handshake.de>
 <mailman.25.1545202642.31622.python-list@python.org>
 <41vpef-2n2.ln1@anthive.com>
Message-ID: <901qef-f53.ln1@anthive.com>

ant wrote:

...

  script was there, but the package was not actually
installed.

  after installing again i get:

=====

(env) me at ant(26)~/src/test$ ngfp
Traceback (most recent call last):
  File "/home/me/src/env/bin/ngfp", line 7, in <module>
    from ngfp import main
ImportError: cannot import name 'main' from 'ngfp' (/home/me/src/salsa/env/lib/python3.7/site-packages/ngfp/__init__.py)

=====

  i'm not sure if this is because my __init__.py is
another directory deeper (my project has a top level
directory with the setup.py in it and then the
ngfp directory with the __init__.py in it which 
contains the following:

=====
name = "ngfp"
=====

  which obviously says nothing about main...
i hate being a newbie.

  but at least i'm making progress.  :)


  ant


From ant at anthive.com  Wed Dec 19 12:22:08 2018
From: ant at anthive.com (ant)
Date: Wed, 19 Dec 2018 12:22:08 -0500
Subject: ah, progress...
References: <n5ooef-a7a.ln1@anthive.com> <87a7l2gg1i.fsf@handshake.de>
 <mailman.25.1545202642.31622.python-list@python.org>
 <41vpef-2n2.ln1@anthive.com> <901qef-f53.ln1@anthive.com>
Message-ID: <0bbqef-u25.ln1@anthive.com>

ant wrote:
> ant wrote:
>
> ...
>
>   script was there, but the package was not actually
> installed.
>
>   after installing again i get:
>
>=====
>
> (env) me at ant(26)~/src/test$ ngfp
> Traceback (most recent call last):
>   File "/home/me/src/env/bin/ngfp", line 7, in <module>
>     from ngfp import main
> ImportError: cannot import name 'main' from 'ngfp' (/home/me/src/salsa/env/lib/python3.7/site-packages/ngfp/__init__.py)
>
>=====
>
>   i'm not sure if this is because my __init__.py is
> another directory deeper (my project has a top level
> directory with the setup.py in it and then the
> ngfp directory with the __init__.py in it which 
> contains the following:
>
>=====
> name = "ngfp"
>=====
>
>   which obviously says nothing about main...
> i hate being a newbie.
>
>   but at least i'm making progress.  :)


hmmm, but my main is defined in ngfp.py


=====

...


def main():
    window = Window(width=cfg.img_pix*(cfg.game_cols+cfg.control_cols+3), height=cfg.img_pix*(cfg.game_rows+2), caption="Ngfp", resizable=True, fullscreen=False, visible=False)
    pyglet.clock.schedule_interval(window.update, 1/120.0) # update at 60Hz
    pyglet.app.run()


if __name__ == "__main__":
    main()


=====


  when i'm in the directory with the source code:


$ python3 ngfp.py


works exactly as it should.


  ant


From oscar.j.benjamin at gmail.com  Wed Dec 19 13:22:21 2018
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Wed, 19 Dec 2018 18:22:21 +0000
Subject: Mask two images with python
In-Reply-To: <c835f128-fa2c-43be-8be9-a3b3baccc749@googlegroups.com>
References: <c835f128-fa2c-43be-8be9-a3b3baccc749@googlegroups.com>
Message-ID: <CAHVvXxSVvNVxZc4+CU6V4544VBDFE4uz+jJF2UMqo2QnnqU7tA@mail.gmail.com>

On Wed, 19 Dec 2018 at 05:42, Umar Yusuf <bintacomputers at gmail.com> wrote:
>
> Hello there,
> How do I supper impose an image design on a transparent png image?
>
> I have tried to use OpenCV's "cv2.bitwise_and" function to no success. I posted the detail question here: https://stackoverflow.com/questions/53791510/python-opencv-mask-and-glow

I don't understand what you're trying to do. Can you show example
input and desired output with example numbers? E.g.:

mask = [
[0, 0, 0, 0],
[0, 1, 1, 0],
...
]

image = [
[12, 32, 45, 56],
...

expected_output = ?

--
Oscar


From brian_christians at hotmail.com  Wed Dec 19 17:32:17 2018
From: brian_christians at hotmail.com (Brian Christiansen)
Date: Wed, 19 Dec 2018 15:32:17 -0700
Subject: polar coordinates?
In-Reply-To: <mailman.28.1545247171.31622.python-list@python.org>
References: <pujfum$dsb$1@dont-email.me> <pvc59k$ics$1@dont-email.me>
 <tqvk1e5shmupotr6flpilbn3k92m836995@4ax.com>
 <mailman.28.1545247171.31622.python-list@python.org>
Message-ID: <pvegtk$2d2$1@dont-email.me>

On 12/19/18 12:19 PM, Dennis Lee Bieber wrote:
> 
> 	I don't see anything in your code that makes any use of
> polar<>rectangular coordinate conversion -- which is what you were
> interested in when starting the thread. You appear to just be rastering a
> 15x15 rectangle (ie: the first 225 digits. If you really want to plot all
> the digits you provide, you should be looping over /them/, not some
> arbitrary X and Y coordinates.
> 
> 
I didn't say I yet had what I need to use polar coordinates for yet 
working, or that I even yet had done anything past some planning in my 
head.  In the numberphile video I linked to, the 3rd (I think) way to 
represent PI (the one professor Grimes says looks like the tiling in a 
Roman bathhouse) and I think the 4th way (the one that is the other 
guy's favorite I think) have only been designed a bit in my head so far. 
  I just think those 2 ways, which no code has been written for yet, 
would be easier to do using polar coordinates.  And yes, I know there 
are ways that I could have used all 1000 digits of PI rather than just 
49 of them.
>> pi1000 =
>> [3,1,4,1,5,9,2,6,5,3,5,8,9,7,9,3,2,3,8,4,6,2,6,4,3,3,8,3,2,7,9,5,0,2,8,
>>            8,4,1,9,7,1,6,9,3,9,9,3,7,5,1,0,
> 		<SNIP>
>>            8,7,6,6,1,1,1,9,5,9,0,9,2,1,6,4,2,0,1,9,8,9]
> 
> 	Pardon me while I go UGH!... That must have been tedious to enter (and
> why double spaced). This may add a few milliseconds in processing (I'll
> explain later), but looks a lot cleaner:
> 
> pi1000 = ("3.1415926535897932384626433832795028"
> 	"8419716939937510582097494459230781"
> 		<SNIP>
> 	"8766111959092164201989")
> 
> 
As for putting in the value for PI, it was actually rather easy.  To get 
the number itself, I just searched for and downloaded a file that had PI 
to 1000 places and copied and pasted it into the editor.  As for the 
commas, at least to my current understanding of python, I thought those 
had to between the elements of an array.  As for putting in the commas, 
I did that by using the "replace all" function in my IDE (or perhaps I 
used gedit, a linux version of notepad) to replace all the "0" with 
"0,", "1" with "1,", etc, so that was not very tedious and was actually 
pretty fast.  As for the double spacing, like I said it seems that 
usenet (or perhaps my usenet viewer) seems to mess with spacing or 
whatever, they do not exist on the actual program on my compputer.

> 	Python will concatenate adjacent strings, so that becomes just one long
> string internally -- if you were to "print pi1000" (Python 2.x syntax) it
> would display:
> 
> 3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117067982148086513282306647093844609550582231725359408128481117450284102701938521105559644622948954930381964428810975665933446128475648233786783165271201909145648566923460348610454326648213393607260249141273724587006606315588174881520920962829254091715364367892590360011330530548820466521384146951941511609433057270365759591953092186117381932611793105118548074462379962749567351885752724891227938183011949129833673362440656643086021394946395224737190702179860943702770539217176293176752384674818467669405132000568127145263560827785771342757789609173637178721468440901224953430146549585371050792279689258923542019956112129021960864034418159813629774771309960518707211349999998372978049951059731732816096318595024459455346908302642522308253344685035261931188171010003137838752886587533208381420617177669147303598253490428755468731159562863882353787593751957781857780532171226806613001927876611195909216420
> 1989
> 
> 
> 	Consider this; I spent way too much time generating the code when I
> should be out shopping for Christmas (watch out for line wrapping):
> 
> -=-=-=-=-
> import math
> 
> import matplotlib as mpl
> import matplotlib.pyplot as plt
> 
> PI1000 = ("3.1415926535897932384626433832795028"
> 	"8419716939937510582097494459230781"
> 	"64062862089986280348253421170679821"
> 	"48086513282306647093844609550582231"
> 	"72535940812848111745028410270193852"
> 	"11055596446229489549303819644288109"
> 	"75665933446128475648233786783165271"
> 	"20190914564856692346034861045432664"
> 	"82133936072602491412737245870066063"
> 	"15588174881520920962829254091715364"
> 	"36789259036001133053054882046652138"
> 	"41469519415116094330572703657595919"
> 	"53092186117381932611793105118548074"
> 	"46237996274956735188575272489122793"
> 	"81830119491298336733624406566430860"
> 	"21394946395224737190702179860943702"
> 	"77053921717629317675238467481846766"
> 	"94051320005681271452635608277857713"
> 	"42757789609173637178721468440901224"
> 	"95343014654958537105079227968925892"
> 	"35420199561121290219608640344181598"
> 	"13629774771309960518707211349999998"
> 	"37297804995105973173281609631859502"
> 	"44594553469083026425223082533446850"
> 	"35261931188171010003137838752886587"
> 	"53320838142061717766914730359825349"
> 	"04287554687311595628638823537875937"
> 	"51957781857780532171226806613001927"
> 	"8766111959092164201989")
> 
> def spiral_plot(digits):
>      #create list of (x, y, color) tuples for the digits
>      #(provided as a string); non-digits get the 11th color
>      #X,Y are coordinates for an spiral
>      x_coords = []
>      y_coords = []
>      xy_color = []
>      mx = 0
>      my = 0
>      for p, d in enumerate(digits):
>          #treating p as angle in degrees and as distance
>          x = math.cos(math.radians(p)) * p
>          y = math.sin(math.radians(p)) * p
>          if d.isdigit():
>              clr = int(d)	#0..9 so
>          else:
>              clr = 10		#10 is the 11th color
>          x_coords.append(x)
>          y_coords.append(y)
>          xy_color.append(clr)
> 
>      plt.scatter(x_coords, y_coords, c=xy_color, cmap=mpl.cm.Spectral,
> s=200)
>      plt.show()
> 
> if __name__ == "__main__":
>      spiral_plot(PI1000)
>      
> -=-=-=-
> 
> 	NOTE: there are no doubt more efficient ways to work this -- for
> example, using NumPy arrays to generate the coordinates en-mass
> 
>
Like I said, or at least think I implied, when I wrote the code I 
posted, I did not even know that matplotlib or numpy even existed, just 
the existence of the "graphics package" graphics.py.  What I have 
working, I do not need polar coordinates for, I need them for what I 
have planned in my head for the indefinite future.

The code you posted appears to make professor Grimes "Roman bathhouse 
tiling" representation of PI (I have not, as of when I typed this 
message cut and pasted it into my IDE to see exactly what happens).  I 
think I will use it, or perhaps an adaptation of it for my "Roman 
bathhouse tiling" representation of PI.  Perhaps I will see if I can 
adapt it to use NumPY, but I suspect that you can do that better than I.



-- 
My Yonkoma: https://www.flickr.com/photos/brian0908/albums/72157680223526176

The E-mail associated with the account is a "spamcatcher" account that I 
got to every couple of months to empty out, and anything sent to it will 
not be seen for probably several months, if it is seen at all.
Brian Christiansen

From dieter at handshake.de  Thu Dec 20 01:58:55 2018
From: dieter at handshake.de (dieter)
Date: Thu, 20 Dec 2018 07:58:55 +0100
Subject: ah, progress...
References: <n5ooef-a7a.ln1@anthive.com> <87a7l2gg1i.fsf@handshake.de>
 <mailman.25.1545202642.31622.python-list@python.org>
 <41vpef-2n2.ln1@anthive.com> <901qef-f53.ln1@anthive.com>
 <0bbqef-u25.ln1@anthive.com>
Message-ID: <871s6cwuog.fsf@handshake.de>

ant <ant at anthive.com> writes:
> ant wrote:
> ...
>> (env) me at ant(26)~/src/test$ ngfp
>> Traceback (most recent call last):
>>   File "/home/me/src/env/bin/ngfp", line 7, in <module>
>>     from ngfp import main
>> ImportError: cannot import name 'main' from 'ngfp' (/home/me/src/salsa/env/lib/python3.7/site-packages/ngfp/__init__.py)
> ...
> hmmm, but my main is defined in ngfp.py

The error message tells you that it looks for "main"
in "...ngfp/__init__.py", not in "ngfp.py".



From pahome.chen at mirlab.org  Thu Dec 20 04:28:40 2018
From: pahome.chen at mirlab.org (lampahome)
Date: Thu, 20 Dec 2018 17:28:40 +0800
Subject: Better way to recontruct a continuous and repeated array with low
 time complexity?
Message-ID: <CAB3eZfsD16XsW1bM+8iXzAJJwbtjcO-31u=b1OXzwfoXMUTC=w@mail.gmail.com>

I write program to do experiment about time series(weekly) with machine
learning.
I record changes of everyday of each ID and Count.
I read the csv as dataset like below:
ID, Count
1,30 // First Day
2,33
3,45
4,11
5,66
7,88
1,32 // 2nd Day
2,35
3,55
4,21
5,36
7,48

I have two array X, y. I want to put ID and Count in X, and put Count of
2nd Day in y.
The element of X and y is corresponding with the ID.
ex: X[0] and y[0] is the value where ID == 1
X[1] and y[1] is the value where ID == 2...etc

So X is like below:
array([
[1,30],
[2,33],
[3,45],
[4,11],
[5,66],
[7,88]
])

y is like below:
array([
[32],
[35],
[55],
[21],
[36],
[48]
])

Program what I write always cost O(n^2) complexity.
Code:

dataframe = pandas.read_csv(path, header=0, engine='python')
dataset = dataframe.dropna().values.astype('float64')
create_data(dataset)

def create_data(dataset):

uni = np.unique(dataset[:,0])
X = np.zeros((7, 2))
y = np.zeros((7, 1))
offset = 0
for i in uni:

index = dataset[:,0] == i

data = dataset[index]

for j in xrange(len(data)-2+1):

X[offset] = data[j]

y[offset] = data[j, 1]

offset += 1

return X, y


I use two for loop and estimate the complexity is O(n^2).

Is there any better way to re-write the code and reduct the time complexity?


From p.f.moore at gmail.com  Thu Dec 20 10:42:43 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 20 Dec 2018 15:42:43 +0000
Subject: Decoding a huge JSON file incrementally
Message-ID: <CACac1F9y=ZMB=8PnNp3g8noaHaP1_22SHSiS045xYoFOrkKYzA@mail.gmail.com>

I'm looking for a way to incrementally decode a JSON file. I know this
has come up before, and in general the problem is not soluble (because
in theory the JSON file could be a single object). In my particular
situation, though, I have a 9GB file containing a top-level array
object, with many elements. So what I could (in theory) do is to parse
an element at a time, yielding them.

The problem is that the stdlib JSON library reads the whole file,
which defeats my purpose. What I'd like is if it would read one
complete element, then just enough far ahead to find out that the
parse was done, and return the object it found (it should probably
also return the "next token", as it can't reliably push it back - I'd
check that it was a comma before proceeding with the next list
element).

I couldn't see a way to get the stdlib json library to read "just as
much as needed" in this way. Did I miss a trick? Or alternatively, is
there a JSON decoder library on PyPI that supports this sort of usage?
I'd rather not have to implement my own JSON parser if I can avoid it.

Thanks,
Paul


From sntshkmr60 at gmail.com  Thu Dec 20 11:37:04 2018
From: sntshkmr60 at gmail.com (sntshkmr60 at gmail.com)
Date: Thu, 20 Dec 2018 08:37:04 -0800 (PST)
Subject: How to generate image with a string?
Message-ID: <75d57495-5bf2-41f4-8e41-95bda2aa3e15@googlegroups.com>

More or less I'm trying to replicate placeholder.com in Python. I'll be using Django in the backend. Given all the parameters, I'm not sure how will I be generating the images.

 * I will be passing width and height of the image and maybe the text and background color. 
 * I also want to center the text vertically and horizontally.
 * The text size will be increased proportionally to the image being generated.

Can someone enlighten my path? What libraries can be capable of doing this?


From rosuav at gmail.com  Thu Dec 20 12:19:17 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 21 Dec 2018 04:19:17 +1100
Subject: Decoding a huge JSON file incrementally
In-Reply-To: <CACac1F9y=ZMB=8PnNp3g8noaHaP1_22SHSiS045xYoFOrkKYzA@mail.gmail.com>
References: <CACac1F9y=ZMB=8PnNp3g8noaHaP1_22SHSiS045xYoFOrkKYzA@mail.gmail.com>
Message-ID: <CAPTjJmo1Q+sgeXPB1n=8=iNU+44tWCmObYHyB=F_UzaSaR1MLA@mail.gmail.com>

On Fri, Dec 21, 2018 at 2:44 AM Paul Moore <p.f.moore at gmail.com> wrote:
>
> I'm looking for a way to incrementally decode a JSON file. I know this
> has come up before, and in general the problem is not soluble (because
> in theory the JSON file could be a single object). In my particular
> situation, though, I have a 9GB file containing a top-level array
> object, with many elements. So what I could (in theory) do is to parse
> an element at a time, yielding them.
>
> The problem is that the stdlib JSON library reads the whole file,
> which defeats my purpose. What I'd like is if it would read one
> complete element, then just enough far ahead to find out that the
> parse was done, and return the object it found (it should probably
> also return the "next token", as it can't reliably push it back - I'd
> check that it was a comma before proceeding with the next list
> element).

It IS possible to do an incremental parse, but for that to work, you
would need to manually strip off the top-level array structure. What
you'd need to use would be this:

https://docs.python.org/3/library/json.html#json.JSONDecoder.raw_decode

It'll parse stuff and then tell you about what's left. Since your data
isn't coming from a ginormous string, but is coming from a file,
you're probably going to need something like this:

def get_stuff_from_file(f):
    buffer = ""
    dec = json.JSONDecoder()
    while "not eof":
        while "no object yet":
            try: obj, pos = dec.raw_decode(buffer)
            except JSONDecodeError: buffer += f.read(1024)
            else: break
        yield obj
        buffer = buffer[pos:].lstrip().lstrip(",")

Proper error handling is left as an exercise for the reader, both in
terms of JSON errors and file errors. Also, the code is completely
untested. Have fun :)

The basic idea is that you keep on grabbing more data till you can
decode an object, then you keep whatever didn't get used up ("pos"
points to whatever didn't get consumed). Algorithmic complexity should
be O(n) as long as your objects are relatively small, and you can
optimize disk access by tuning your buffer size to be at least the
average size of an object.

Hope that helps.

ChrisA


From p.f.moore at gmail.com  Thu Dec 20 12:26:46 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 20 Dec 2018 17:26:46 +0000
Subject: Decoding a huge JSON file incrementally
In-Reply-To: <CAPTjJmo1Q+sgeXPB1n=8=iNU+44tWCmObYHyB=F_UzaSaR1MLA@mail.gmail.com>
References: <CACac1F9y=ZMB=8PnNp3g8noaHaP1_22SHSiS045xYoFOrkKYzA@mail.gmail.com>
 <CAPTjJmo1Q+sgeXPB1n=8=iNU+44tWCmObYHyB=F_UzaSaR1MLA@mail.gmail.com>
Message-ID: <CACac1F_cDMcAc0KRe5GB6gq+Dz10A7hzLZLuM159Z5POWci__Q@mail.gmail.com>

On Thu, 20 Dec 2018 at 17:22, Chris Angelico <rosuav at gmail.com> wrote:
>
> On Fri, Dec 21, 2018 at 2:44 AM Paul Moore <p.f.moore at gmail.com> wrote:
> >
> > I'm looking for a way to incrementally decode a JSON file. I know this
> > has come up before, and in general the problem is not soluble (because
> > in theory the JSON file could be a single object). In my particular
> > situation, though, I have a 9GB file containing a top-level array
> > object, with many elements. So what I could (in theory) do is to parse
> > an element at a time, yielding them.
> >
> > The problem is that the stdlib JSON library reads the whole file,
> > which defeats my purpose. What I'd like is if it would read one
> > complete element, then just enough far ahead to find out that the
> > parse was done, and return the object it found (it should probably
> > also return the "next token", as it can't reliably push it back - I'd
> > check that it was a comma before proceeding with the next list
> > element).
>
> It IS possible to do an incremental parse, but for that to work, you
> would need to manually strip off the top-level array structure. What
> you'd need to use would be this:
>
> https://docs.python.org/3/library/json.html#json.JSONDecoder.raw_decode
>
> It'll parse stuff and then tell you about what's left. Since your data
> isn't coming from a ginormous string, but is coming from a file,
> you're probably going to need something like this:
>
> def get_stuff_from_file(f):
>     buffer = ""
>     dec = json.JSONDecoder()
>     while "not eof":
>         while "no object yet":
>             try: obj, pos = dec.raw_decode(buffer)
>             except JSONDecodeError: buffer += f.read(1024)
>             else: break
>         yield obj
>         buffer = buffer[pos:].lstrip().lstrip(",")
>
> Proper error handling is left as an exercise for the reader, both in
> terms of JSON errors and file errors. Also, the code is completely
> untested. Have fun :)
>
> The basic idea is that you keep on grabbing more data till you can
> decode an object, then you keep whatever didn't get used up ("pos"
> points to whatever didn't get consumed). Algorithmic complexity should
> be O(n) as long as your objects are relatively small, and you can
> optimize disk access by tuning your buffer size to be at least the
> average size of an object.
>
> Hope that helps.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list


From p.f.moore at gmail.com  Thu Dec 20 12:30:30 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 20 Dec 2018 17:30:30 +0000
Subject: Decoding a huge JSON file incrementally
In-Reply-To: <CAPTjJmo1Q+sgeXPB1n=8=iNU+44tWCmObYHyB=F_UzaSaR1MLA@mail.gmail.com>
References: <CACac1F9y=ZMB=8PnNp3g8noaHaP1_22SHSiS045xYoFOrkKYzA@mail.gmail.com>
 <CAPTjJmo1Q+sgeXPB1n=8=iNU+44tWCmObYHyB=F_UzaSaR1MLA@mail.gmail.com>
Message-ID: <CACac1F8t+Va-gxHLR1a_y0-YhBUa3dAo+Fe4qCkO8Hu-BuNvtA@mail.gmail.com>

(Sorry, hit "Send" too soon on the last try!)

On Thu, 20 Dec 2018 at 17:22, Chris Angelico <rosuav at gmail.com> wrote:
>
> On Fri, Dec 21, 2018 at 2:44 AM Paul Moore <p.f.moore at gmail.com> wrote:
> >
> > I'm looking for a way to incrementally decode a JSON file. I know this
> > has come up before, and in general the problem is not soluble (because
> > in theory the JSON file could be a single object). In my particular
> > situation, though, I have a 9GB file containing a top-level array
> > object, with many elements. So what I could (in theory) do is to parse
> > an element at a time, yielding them.
> >
> > The problem is that the stdlib JSON library reads the whole file,
> > which defeats my purpose. What I'd like is if it would read one
> > complete element, then just enough far ahead to find out that the
> > parse was done, and return the object it found (it should probably
> > also return the "next token", as it can't reliably push it back - I'd
> > check that it was a comma before proceeding with the next list
> > element).
>
> It IS possible to do an incremental parse, but for that to work, you
> would need to manually strip off the top-level array structure. What
> you'd need to use would be this:
>
> https://docs.python.org/3/library/json.html#json.JSONDecoder.raw_decode
>
> It'll parse stuff and then tell you about what's left. Since your data
> isn't coming from a ginormous string, but is coming from a file,
> you're probably going to need something like this:
>
> def get_stuff_from_file(f):
>     buffer = ""
>     dec = json.JSONDecoder()
>     while "not eof":
>         while "no object yet":
>             try: obj, pos = dec.raw_decode(buffer)
>             except JSONDecodeError: buffer += f.read(1024)
>             else: break
>         yield obj
>         buffer = buffer[pos:].lstrip().lstrip(",")

Ah, right. I'd found that function, but as it took input from a string
rather than a file-like object, I'd dismissed it. I didn't think of
decoding partial reads. That's a nice trick, thanks!

> Proper error handling is left as an exercise for the reader, both in
> terms of JSON errors and file errors. Also, the code is completely
> untested. Have fun :)

Yeah, once you have the insight that you can attempt to parse a block
at a time, the rest is just a "simple matter of programming" :-)

> The basic idea is that you keep on grabbing more data till you can
> decode an object, then you keep whatever didn't get used up ("pos"
> points to whatever didn't get consumed). Algorithmic complexity should
> be O(n) as long as your objects are relatively small, and you can
> optimize disk access by tuning your buffer size to be at least the
> average size of an object.

Got it, thanks.

> Hope that helps.

Yes it does, a lot. Much appreciated.
Paul


From rosuav at gmail.com  Thu Dec 20 12:59:34 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 21 Dec 2018 04:59:34 +1100
Subject: Decoding a huge JSON file incrementally
In-Reply-To: <CACac1F8t+Va-gxHLR1a_y0-YhBUa3dAo+Fe4qCkO8Hu-BuNvtA@mail.gmail.com>
References: <CACac1F9y=ZMB=8PnNp3g8noaHaP1_22SHSiS045xYoFOrkKYzA@mail.gmail.com>
 <CAPTjJmo1Q+sgeXPB1n=8=iNU+44tWCmObYHyB=F_UzaSaR1MLA@mail.gmail.com>
 <CACac1F8t+Va-gxHLR1a_y0-YhBUa3dAo+Fe4qCkO8Hu-BuNvtA@mail.gmail.com>
Message-ID: <CAPTjJmreBBVNoQkPSbGrLC+b9=Q9Hr-EbEVn2y9u-vvoj65gfQ@mail.gmail.com>

On Fri, Dec 21, 2018 at 4:30 AM Paul Moore <p.f.moore at gmail.com> wrote:
> On Thu, 20 Dec 2018 at 17:22, Chris Angelico <rosuav at gmail.com> wrote:
> > Proper error handling is left as an exercise for the reader, both in
> > terms of JSON errors and file errors. Also, the code is completely
> > untested. Have fun :)
>
> Yeah, once you have the insight that you can attempt to parse a block
> at a time, the rest is just a "simple matter of programming" :-)

Perfect! I can leave it all in your capable hands then. Wasn't sure
how much I'd need to explain there, and it sounds like I
overestimated. :) Have fun with it!

Something to be aware of: exception raising in PYthon is somewhat
expensive, so it may be worth tuning your chunk size upwards a bit to
compensate (keep a bit more in memory, fail the parse less
frequently). Would need to measure. Of course, it's entirely possible
that it won't even matter, and your parse time will be utterly
dominated by disk read times.

ChrisA


From rosuav at gmail.com  Thu Dec 20 13:04:58 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 21 Dec 2018 05:04:58 +1100
Subject: How to generate image with a string?
In-Reply-To: <75d57495-5bf2-41f4-8e41-95bda2aa3e15@googlegroups.com>
References: <75d57495-5bf2-41f4-8e41-95bda2aa3e15@googlegroups.com>
Message-ID: <CAPTjJmoHAkJf0wv9VbKZ7xvrymYy++O-HdNOfthXP4EfROmJbQ@mail.gmail.com>

On Fri, Dec 21, 2018 at 4:50 AM <sntshkmr60 at gmail.com> wrote:
>
> More or less I'm trying to replicate placeholder.com in Python. I'll be using Django in the backend. Given all the parameters, I'm not sure how will I be generating the images.
>
>  * I will be passing width and height of the image and maybe the text and background color.
>  * I also want to center the text vertically and horizontally.
>  * The text size will be increased proportionally to the image being generated.
>
> Can someone enlighten my path? What libraries can be capable of doing this?

Django is irrelevant here, so what you're trying to do is use Python
code to generate an image containing some text. I don't know the
specifics of that, but I know where I'd go looking, and that's Pillow.
(No, I don't mean that I'd sleep on it, although that IS often a good
idea.) Here's the docs:

https://pillow.readthedocs.io/en/5.3.x/

The ImageDraw and ImageFont modules look promising here.

I would recommend creating a stand-alone script (outside of Django and
the whole web server system) that generates and saves an image, and
only bring it into your web server once it's working.

ChrisA


From python at mrabarnett.plus.com  Thu Dec 20 13:20:32 2018
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 20 Dec 2018 18:20:32 +0000
Subject: How to generate image with a string?
In-Reply-To: <75d57495-5bf2-41f4-8e41-95bda2aa3e15@googlegroups.com>
References: <75d57495-5bf2-41f4-8e41-95bda2aa3e15@googlegroups.com>
Message-ID: <87d30bd2-eae5-fbdc-f48c-bf0153ceabba@mrabarnett.plus.com>

On 2018-12-20 16:37, sntshkmr60 at gmail.com wrote:
> More or less I'm trying to replicate placeholder.com in Python. I'll be using Django in the backend. Given all the parameters, I'm not sure how will I be generating the images.
> 
>   * I will be passing width and height of the image and maybe the text and background color.
>   * I also want to center the text vertically and horizontally.
>   * The text size will be increased proportionally to the image being generated.
> 
> Can someone enlighten my path? What libraries can be capable of doing this?
> 
Have a look at the Pillow module. It's available from PyPI:

https://pypi.org/project/Pillow/


From tjreedy at udel.edu  Thu Dec 20 13:52:29 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 20 Dec 2018 13:52:29 -0500
Subject: Decoding a huge JSON file incrementally
In-Reply-To: <CACac1F9y=ZMB=8PnNp3g8noaHaP1_22SHSiS045xYoFOrkKYzA@mail.gmail.com>
References: <CACac1F9y=ZMB=8PnNp3g8noaHaP1_22SHSiS045xYoFOrkKYzA@mail.gmail.com>
Message-ID: <pvgo9c$j7q$1@blaine.gmane.org>

On 12/20/2018 10:42 AM, Paul Moore wrote:
> I'm looking for a way to incrementally decode a JSON file. I know this
> has come up before, and in general the problem is not soluble (because
> in theory the JSON file could be a single object).

AFAIK, a JSON file always represents a single JSON item and is 
translated (decoded) to a single Python object.

The json encoder has a iterencode method, but the decoder does not have 
an iterdecode method.  I think it plausibly should have one that would 
iterate through a top-level list or dict (JSON array or object).

> In my particular
> situation, though, I have a 9GB file containing a top-level array
> object, with many elements. So what I could (in theory) do is to parse
> an element at a time, yielding them.

So your file format, not worrying about whitespace and the possible lack 
of ',' after the last item, is '[' (item ',')* ']'.  You want to skip 
over the '[' instead of creating an empty list, then yield each item 
rather than appending to the list.

> The problem is that the stdlib JSON library reads the whole file,
> which defeats my purpose. What I'd like is if it would read one
> complete element, then just enough far ahead to find out that the
> parse was done, and return the object it found (it should probably
> also return the "next token", as it can't reliably push it back - I'd
> check that it was a comma before proceeding with the next list
> element).

I looked at json.decode and json.scanner.  After reading the whole file 
into a string, json decodes the string an item at a time with a 
scan_once(string, index) function that finds the end of the first item 
in the string.  It then returns the decoded item and the index of where 
to continue scanning for the next item. If the string does not begin 
with a complete representation of an item, json.decode.JSONDecodeError 
is raised.

So I believe you could fairly easily write a function roughly as follows.
   open file and read and check the initial '['
   s = ''; idx = 0
   scanner = make_scanner(context)
   # I did not figure out what 'context' should be
   while more in file:
     s += large chunk
     try:
       ob, idx = scanner.scan_once(s, idx)
       yield ob
     except JSONDecodeError as e:
       check that problem is incompleteness rather than bad format


-- 
Terry Jan Reedy



From fetchinson at googlemail.com  Thu Dec 20 16:48:36 2018
From: fetchinson at googlemail.com (Fetchinson .)
Date: Thu, 20 Dec 2018 22:48:36 +0100
Subject: building 3.7.1 from source, _ctypes and libffi troubles
Message-ID: <CADjSo4SfUXb5oa9sHbFbV5GKAWdrJeMyCftVKO4xTdLmTpd+uw@mail.gmail.com>

Hi all, I'm trying to build 3.7.1 from source and having trouble with
libffi and _ctypes. I'm on linux and have installed libffi also from
source to a custom location:

$HOME/opt/lib64/libffi.so.6.0.4
$HOME/opt/lib64/libffi.a
$HOME/opt/lib64/libffi.la
$HOME/opt/lib64/libffi.so.6
$HOME/opt/lib64/libffi.so
$HOME/opt/lib/pkgconfig/libffi.pc
$HOME/opt/lib/libffi-3.2.1/include/ffi.h
$HOME/opt/lib/libffi-3.2.1/include/ffitarget.h
$HOME/opt/share/info/libffi.info

I have no idea why the installation is somewhat scattered, these are
the default locations that were created by

./configure --prefix=$HOME/opt

of the source libffi.

In any case, just to be sure, I've copied the header files to

$HOME/opt/include/ffi.h
$HOME/opt/include/ffitarget.h

Looks like pkg-config works as intended:

[fetch at fetch opt]$ pkg-config --libs libffi
-L/home/fetch/opt/lib/../lib64 -lffi

[fetch at fetch opt]$ pkg-config --cflags libffi
-I/home/fetch/opt/lib/libffi-3.2.1/include

And as far as I know pkg-config is used by python's configure script
so everything should be fine. I also set
LD_LIBRARY_PATH=/home/fetch/opt/lib:/home/fetch/opt/lib64 and also
C_INCLUDE_PATH=/home/fetch/opt/include

And still I get

Failed to build these modules:
_ctypes

from make when I try to build python 3.7.1 Also tried adding
--with-system-ffi to ./configure but that didn't help either.

Does anyone have an idea what's wrong? I've seen some related bug reports, e.g.

https://bugs.python.org/issue35170

but these were typically solved by the OP installing libffi from the
repository of his/her distro. Note that in my case I must compile
libffi from source.

Any ideas?

Thanks a lot,
Daniel





-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From fetchinson at googlemail.com  Thu Dec 20 18:33:56 2018
From: fetchinson at googlemail.com (Fetchinson .)
Date: Fri, 21 Dec 2018 00:33:56 +0100
Subject: building 3.7.1 from source, _ctypes and libffi troubles
In-Reply-To: <CADjSo4SfUXb5oa9sHbFbV5GKAWdrJeMyCftVKO4xTdLmTpd+uw@mail.gmail.com>
References: <CADjSo4SfUXb5oa9sHbFbV5GKAWdrJeMyCftVKO4xTdLmTpd+uw@mail.gmail.com>
Message-ID: <CADjSo4SA3rm_=oFpxm3e6MTLaM3CnPycT6v2KLEwJjzhFwzG_w@mail.gmail.com>

On 12/20/18, Fetchinson . <fetchinson at googlemail.com> wrote:
> Hi all, I'm trying to build 3.7.1 from source and having trouble with
> libffi and _ctypes. I'm on linux and have installed libffi also from
> source to a custom location:
>
> $HOME/opt/lib64/libffi.so.6.0.4
> $HOME/opt/lib64/libffi.a
> $HOME/opt/lib64/libffi.la
> $HOME/opt/lib64/libffi.so.6
> $HOME/opt/lib64/libffi.so
> $HOME/opt/lib/pkgconfig/libffi.pc
> $HOME/opt/lib/libffi-3.2.1/include/ffi.h
> $HOME/opt/lib/libffi-3.2.1/include/ffitarget.h
> $HOME/opt/share/info/libffi.info
>
> I have no idea why the installation is somewhat scattered, these are
> the default locations that were created by
>
> ./configure --prefix=$HOME/opt
>
> of the source libffi.
>
> In any case, just to be sure, I've copied the header files to
>
> $HOME/opt/include/ffi.h
> $HOME/opt/include/ffitarget.h
>
> Looks like pkg-config works as intended:
>
> [fetch at fetch opt]$ pkg-config --libs libffi
> -L/home/fetch/opt/lib/../lib64 -lffi
>
> [fetch at fetch opt]$ pkg-config --cflags libffi
> -I/home/fetch/opt/lib/libffi-3.2.1/include
>
> And as far as I know pkg-config is used by python's configure script
> so everything should be fine. I also set
> LD_LIBRARY_PATH=/home/fetch/opt/lib:/home/fetch/opt/lib64 and also
> C_INCLUDE_PATH=/home/fetch/opt/include
>
> And still I get
>
> Failed to build these modules:
> _ctypes
>
> from make when I try to build python 3.7.1 Also tried adding
> --with-system-ffi to ./configure but that didn't help either.
>
> Does anyone have an idea what's wrong? I've seen some related bug reports,
> e.g.
>
> https://bugs.python.org/issue35170
>
> but these were typically solved by the OP installing libffi from the
> repository of his/her distro. Note that in my case I must compile
> libffi from source.
>
> Any ideas?
>
> Thanks a lot,
> Daniel

I just found a bug report from the end of September detailing the
exact same problem:

https://bugs.python.org/issue34823

It's still open with no comments at all :(

Is there a chance this will be fixed?

Summarizing: the problem is that python can't be build on linux from
source if libffi is installed to non-standard directories.

Cheers,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown


From ant at anthive.com  Fri Dec 21 01:06:24 2018
From: ant at anthive.com (ant)
Date: Fri, 21 Dec 2018 01:06:24 -0500
Subject: ah, progress...
References: <n5ooef-a7a.ln1@anthive.com> <87a7l2gg1i.fsf@handshake.de>
 <mailman.25.1545202642.31622.python-list@python.org>
 <41vpef-2n2.ln1@anthive.com> <901qef-f53.ln1@anthive.com>
 <0bbqef-u25.ln1@anthive.com> <871s6cwuog.fsf@handshake.de>
 <mailman.30.1545289149.31622.python-list@python.org>
Message-ID: <0gcuef-sfc.ln1@anthive.com>

dieter wrote:
> ant <ant at anthive.com> writes:
>> ant wrote:
>> ...
>>> (env) me at ant(26)~/src/test$ ngfp
>>> Traceback (most recent call last):
>>>   File "/home/me/src/env/bin/ngfp", line 7, in <module>
>>>     from ngfp import main
>>> ImportError: cannot import name 'main' from 'ngfp' (/home/me/src/salsa/env/lib/python3.7/site-packages/ngfp/__init__.py)
>> ...
>> hmmm, but my main is defined in ngfp.py
>
> The error message tells you that it looks for "main"
> in "...ngfp/__init__.py", not in "ngfp.py".

  i took an examply python program called afew and
edited it down to work for a very simple program
which i then took my other code and put it in that
directory structure to make sure i wasn't doing
anything strange, but i still ended up where 
running the program one way works and running it
the other way it seems to miss something important
which i don't understand.

  in order to get this far below i had to edit each
file and put a try: except: around each import
statment checking if the module could be found
like (as an example):

try:
    import config as cfg
except:
    import frog.config as cfg


this is very frustrating because the following makes 
no sense to me:

when i run the code within the directory like this i get:
=====  this works as it should
(env) me at ant(30)~/src/salsa/frog/frog$ python3 ngfp.py 
current_dir :  /home/me/src/salsa/frog/frog
(env) me at ant(31)~/src/salsa/frog/frog$ 
=====

when i pack it up and then install it using (and then run it via 
the script):
===== it doesn't work, but the directory is the same...?????
(env) me at ant(150)~/src/salsa/test$ pip3 --no-cache-dir install -e ../frog/ --force-reinstall
Obtaining file:///home/me/src/salsa/frog
Requirement already satisfied: pyglet>=1.3.0 in /home/me/src/salsa/env/lib/python3.7/site-packages (from frog==1.0.0)
Requirement already satisfied: future in /home/me/src/salsa/env/lib/python3.7/site-packages (from pyglet>=1.3.0->frog==1.0.0)
Installing collected packages: frog
  Found existing installation: frog 1.0.0
    Uninstalling frog-1.0.0:
      Successfully uninstalled frog-1.0.0
  Running setup.py develop for frog
Successfully installed frog
(env) me at ant(151)~/src/salsa/test$ frog
current_dir :  /home/me/src/salsa/frog/frog
Traceback (most recent call last):
  File "/home/me/src/salsa/env/bin/frog", line 11, in <module>
    load_entry_point('frog', 'console_scripts', 'frog')()
  File "/home/me/src/salsa/frog/frog/commands.py", line 12, in main
    inner_main()
  File "/home/me/src/salsa/frog/frog/ngfp.py", line 284, in main
    window = Window(width=cfg.img_pix*(cfg.game_cols+cfg.control_cols+3), height=cfg.img_pix*(cfg.game_rows+2), caption="Ngfp", resizable=True, fullscreen=False, visible=False)
  File "/home/me/src/salsa/frog/frog/ngfp.py", line 63, in __init__
    MyInitStuff (self)
  File "/home/me/src/salsa/frog/frog/my_init.py", line 113, in MyInitStuff
    self.game_bg_image  = pyglet.image.load("png/mirrors/00_bg.png")
  File "/home/me/src/salsa/env/lib/python3.7/site-packages/pyglet/image/__init__.py", line 180, in load
    file = open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'png/mirrors/00_bg.png'
(env) me at ant(152)~/src/salsa/test$ 
=====

  the paths are exactly the same when i print the current
directory.

  the png directory is there and the image is there.

  ugh, time to sleep...


  ant


From dieter at handshake.de  Fri Dec 21 01:55:04 2018
From: dieter at handshake.de (dieter)
Date: Fri, 21 Dec 2018 07:55:04 +0100
Subject: ah, progress...
References: <n5ooef-a7a.ln1@anthive.com> <87a7l2gg1i.fsf@handshake.de>
 <mailman.25.1545202642.31622.python-list@python.org>
 <41vpef-2n2.ln1@anthive.com> <901qef-f53.ln1@anthive.com>
 <0bbqef-u25.ln1@anthive.com> <871s6cwuog.fsf@handshake.de>
 <mailman.30.1545289149.31622.python-list@python.org>
 <0gcuef-sfc.ln1@anthive.com>
Message-ID: <87h8f7v06v.fsf@handshake.de>

ant <ant at anthive.com> writes:
> ...
>   in order to get this far below i had to edit each
> file and put a try: except: around each import
> statment checking if the module could be found
> like (as an example):
>
> try:
>     import config as cfg
> except:
>     import frog.config as cfg

Is "frog" the package, you want to install?
Then always use "import frog.config ...".

Python locates (top level) packages/modules via the so
called "Python path" (--> "sys.path"). When
you run a script, the "Python path" is extended by the
directory the script resides in -- thus a script typically
can import "local" packages/module (residing in its directory).
However, it is good practise to always access installed
packages via their "official package path" (which
in your case likely means "frog.config").


When you develop a package, it may not yet be installed.
This might hinder you to import in the "official" way.

I work around this by using "setuptools" and its "setup"
function in my "setup.py". "setuptools" supports the
command "develop" (activated via "python setup.py develop")
which "installs" the package in a way that it refers to
the developped sources. This way, I can always
import in the "official" way and nevertheless all
my changes are effective immediately.

Another approach would be to lay out your package in such
a way that it refects the installed structure and
for testing put your script in the directory containing
your top level packages (or extend "sys.path" with this directory).



From anthony.flury at btinternet.com  Fri Dec 21 03:06:23 2018
From: anthony.flury at btinternet.com (Anthony Flury)
Date: Fri, 21 Dec 2018 08:06:23 +0000
Subject: Side by side comparison - CPython, nuitka, PyPy
Message-ID: <5f2ca83d-b1c0-facf-8b92-c5f9805e74c8@btinternet.com>

I thought I would look at a side by side comparison of CPython, nuitka 
and PyPy

*The functionality under test**
*

I have a library (called primelib) which implements a Sieve of 
Erathoneses in pure Python - it was orginally written as part of my 
project Euler attempts

Not only does it build a sieve to test primality, it also builds an 
iterable list of primes, and has functionality to calculate the prime 
factors (and exponents) and also calculate all divisors of a given 
integer (up to the size of the sieve).

To test the primelib there is a simple harness which :

  * Builds a sieve for integers from 2 to 104729 (104729 is the 10,000th
    prime number)
  * Using a pre-built list from primes.utm.edu -
      o For every integer from 2 to 104729 the prime sieve and pre-built
        list agree on the primality or non-primality
      o confirm that the list of ALL primes identified by the sieve is
        the same as the pre-built list.
      o For every integer from 2 to 104729, get primelib to generate the
        prime factors and exponents - and comfirm that they multiply up
        to the expected integer
      o For every integer from 2 to 104729 get primelib to generate the
        divisors on the integer, and confirm that each divisor does
        divide cleanly into the integer

The Sieve is rebuilt between each test, there is no caching of data 
between test cases, so the test harness forces a lot of recalculations.

I have yet to convert primelib to be Python 3 compatible.

Exactly the same test harness was run in all 3 cases :

  * Under CPython 2.7.15, the execution of the test harness took around
    75 seconds to execute over 5 runs - fastest 73, slowest 78.
  * Under Nuitka 0.6, the execution of the test harness after compiler
    took around 85 seconds over 5 runes, fastest 84, slowest 86.
  * Under PyPy, the execution of the test harness took 4.9 seconds on
    average over 5 runs, fastest 4.79, slowest 5.2

I was very impressed at the execution time improvement under PyPy, and a 
little surprised about the lack of improvement under Nuitka.

I know Nuitka is a work in progress, but given that Nuitka compiles 
Python to C code I would have expected some level of gain, especially in 
a maths heavy implementation.

This comparison is provided for information only, and is not intended as 
any form of formal benchmark. I don't claim that primelib is as 
efficient as it could be - although every effort was made to try to make 
it as fast as I could.

-- 
Anthony Flury
*Email* : anthony.flury at btinternet.com <mailto:Anthony.flury at btinternet.com>
*Twitter* : @TonyFlury <https://twitter.com/TonyFlury/>


From bintacomputers at gmail.com  Fri Dec 21 04:25:57 2018
From: bintacomputers at gmail.com (Umar Yusuf)
Date: Fri, 21 Dec 2018 01:25:57 -0800 (PST)
Subject: Mask two images with python
In-Reply-To: <mailman.27.1545243755.31622.python-list@python.org>
References: <c835f128-fa2c-43be-8be9-a3b3baccc749@googlegroups.com>
 <CAHVvXxSVvNVxZc4+CU6V4544VBDFE4uz+jJF2UMqo2QnnqU7tA@mail.gmail.com>
 <mailman.27.1545243755.31622.python-list@python.org>
Message-ID: <db531940-7588-4c2e-9ad1-0eb322d660da@googlegroups.com>

On Wednesday, 19 December 2018 19:22:51 UTC+1, Oscar Benjamin  wrote:
> On Wed, 19 Dec 2018 at 05:42, Umar Yusuf <bin... at gmail.com> wrote:
> >
> > Hello there,
> > How do I supper impose an image design on a transparent png image?
> >
> > I have tried to use OpenCV's "cv2.bitwise_and" function to no success. I posted the detail question here: https://stackoverflow.com/questions/53791510/python-opencv-mask-and-glow
> 
> I don't understand what you're trying to do. Can you show example
> input and desired output with example numbers? E.g.:
> 
> mask = [
> [0, 0, 0, 0],
> [0, 1, 1, 0],
> ...
> ]
> 
> image = [
> [12, 32, 45, 56],
> ...
> 
> expected_output = ?
> 
> --
> Oscar


Hello Oscar,
Thank you for your time.. Here is attached is an example set of the images: https://drive.google.com/file/d/1hyiWswx4GCZQDepXZjktq2zgM_LBbxXt/view?usp=sharing basically, it is "design + mask and glow = result"






From oscar.j.benjamin at gmail.com  Fri Dec 21 07:31:49 2018
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 21 Dec 2018 12:31:49 +0000
Subject: Mask two images with python
In-Reply-To: <db531940-7588-4c2e-9ad1-0eb322d660da@googlegroups.com>
References: <c835f128-fa2c-43be-8be9-a3b3baccc749@googlegroups.com>
 <CAHVvXxSVvNVxZc4+CU6V4544VBDFE4uz+jJF2UMqo2QnnqU7tA@mail.gmail.com>
 <mailman.27.1545243755.31622.python-list@python.org>
 <db531940-7588-4c2e-9ad1-0eb322d660da@googlegroups.com>
Message-ID: <CAHVvXxSXrC6h3dnS3DOQAjcV4hQ_riPKFG4tq+kR5Eac=31J5w@mail.gmail.com>

On Fri, 21 Dec 2018 at 09:32, Umar Yusuf <bintacomputers at gmail.com> wrote:
>
> On Wednesday, 19 December 2018 19:22:51 UTC+1, Oscar Benjamin  wrote:
> > On Wed, 19 Dec 2018 at 05:42, Umar Yusuf <bin... at gmail.com> wrote:
> > >
> > > Hello there,
> > > How do I supper impose an image design on a transparent png image?
> > >
> > > I have tried to use OpenCV's "cv2.bitwise_and" function to no success. I posted the detail question here: https://stackoverflow.com/questions/53791510/python-opencv-mask-and-glow
> >
> > I don't understand what you're trying to do. Can you show example
> > input and desired output with example numbers? E.g.:
> >
> > mask = [
> > [0, 0, 0, 0],
> > [0, 1, 1, 0],
> > ...
> > ]
> >
> > image = [
> > [12, 32, 45, 56],
> > ...
> >
> > expected_output = ?
>
> Hello Oscar,
> Thank you for your time.. Here is attached is an example set of the images: https://drive.google.com/file/d/1hyiWswx4GCZQDepXZjktq2zgM_LBbxXt/view?usp=sharing basically, it is "design + mask and glow = result"

The reason your question hasn't been answered yet either here or on
StackOverflow is because you need to to take more time to explain it
better. I have suggested above a way to explain it better. If you can
show a clear example of what the input and output pixels are supposed
to be then I'm sure that someone would be able to say how to do it.

>From your description I don't understand unambiguously what you're
trying to do though.

--
Oscar


From ant at anthive.com  Fri Dec 21 09:27:12 2018
From: ant at anthive.com (ant)
Date: Fri, 21 Dec 2018 09:27:12 -0500
Subject: Mask two images with python
References: <c835f128-fa2c-43be-8be9-a3b3baccc749@googlegroups.com>
 <CAHVvXxSVvNVxZc4+CU6V4544VBDFE4uz+jJF2UMqo2QnnqU7tA@mail.gmail.com>
 <mailman.27.1545243755.31622.python-list@python.org>
 <db531940-7588-4c2e-9ad1-0eb322d660da@googlegroups.com>
Message-ID: <0r9vef-4h1.ln1@anthive.com>

Umar Yusuf wrote:
...
> Hello Oscar,
> Thank you for your time.. Here is attached is an example set of the images: https://drive.google.com/file/d/1hyiWswx4GCZQDepXZjktq2zgM_LBbxXt/view?usp=sharing basically, it is "design + mask and glow = result"

  that didn't work for me at all...

  are the images .png or some other format that
allows transparency?

  is it critical that it be done via Python or do you
just want it done at all?  :)

  gimp and other tools (Graphics Magic) will allow 
this to be done interactively or via the command 
line.

  if it has to be done in python i'm only familiar
with pyglet and could do it easily enough that way
(sorry i'm a newbie about other ways in python :) ).

make a sprite in a certain location in a window with
the background image and then make another sprite with
the next layer on top of that one, one at a time, each
with it's own batch id.

then when it appears how you like it you can grab a
screen shot via:

====
The following example shows how to grab a screenshot of your application window:

pyglet.image.get_buffer_manager().get_color_buffer().save('screenshot.png')

Note that images can only be saved in the PNG format unless the Pillow library is installed.
=====

  if you want an example of multiple layers i have
my work in progress that does exactly this sort of 
thing (except i never save an image yet - that could
be easily added as above).  you're welcome to take
the code and use it however you like:


  https://salsa.debian.org/ant-guest/gfpoken-in-python


  ant


From ant at anthive.com  Fri Dec 21 10:04:22 2018
From: ant at anthive.com (ant)
Date: Fri, 21 Dec 2018 10:04:22 -0500
Subject: ah, progress...
References: <n5ooef-a7a.ln1@anthive.com> <87a7l2gg1i.fsf@handshake.de>
 <mailman.25.1545202642.31622.python-list@python.org>
 <41vpef-2n2.ln1@anthive.com> <901qef-f53.ln1@anthive.com>
 <0bbqef-u25.ln1@anthive.com> <871s6cwuog.fsf@handshake.de>
 <mailman.30.1545289149.31622.python-list@python.org>
 <0gcuef-sfc.ln1@anthive.com> <87h8f7v06v.fsf@handshake.de>
 <mailman.44.1545375314.31622.python-list@python.org>
Message-ID: <m0cvef-mr1.ln1@anthive.com>

dieter wrote:
> ant <ant at anthive.com> writes:
>> ...
>>   in order to get this far below i had to edit each
>> file and put a try: except: around each import
>> statment checking if the module could be found
>> like (as an example):
>>
>> try:
>>     import config as cfg
>> except:
>>     import frog.config as cfg
>
> Is "frog" the package, you want to install?
> Then always use "import frog.config ...".

  frog is my temporary adaptation of my project
to test out how to do this without actually making
all those changes until i'm sure it is doing what
i want.

  frog is a package as i understand it.  here is
the structure:

=====
(env) me at ant(15)~/src/salsa/frog$ find . -print | grep -v ".git" | grep -v "__pycache__" | grep -v "dist" | grep -v "build" | sort
.
./frog
./frog/active.py
./frog/background.py
./frog/board.py
./frog/commands.py
./frog/config.py
./frog/dialog.py
./frog/doc
./frog/doc/ngfp.6
./frog.egg-info
./frog.egg-info/dependency_links.txt
./frog.egg-info/entry_points.txt
./frog.egg-info/PKG-INFO
./frog.egg-info/requires.txt
./frog.egg-info/SOURCES.txt
./frog.egg-info/top_level.txt
./frog/history.py
./frog/__init__.py
./frog/labels.py
./frog/__main__.py
./frog/marbles.py
./frog/my_init.py
./frog/ngfp.py
./frog/png
./frog/png/arrows
./frog/png/arrows/picDDownW.png
./frog/png/arrows/picDLeftW.png
...80 lines skipped...
./frog/png/misc/sink_inv.png
./frog/png/misc/sink_orig.png
./frog/randboard.py
./LICENSE
./LICENSE.GPL
./MANIFEST.in
./NOTICE
./README.md
./setup.cfg
./setup.py
=====


> Python locates (top level) packages/modules via the so
> called "Python path" (--> "sys.path"). When
> you run a script, the "Python path" is extended by the
> directory the script resides in -- thus a script typically
> can import "local" packages/module (residing in its directory).
> However, it is good practise to always access installed
> packages via their "official package path" (which
> in your case likely means "frog.config").

  this is the first i'm hearing of a path being
extended.  why isn't that reflected when i print the
current working directory when the program is running
(in both cases the paths show identical)?

  this is also the first time i'm hearing of "official
package path".

  there are likely the details i'm beating my head
against...  thank you.  :)


> When you develop a package, it may not yet be installed.
> This might hinder you to import in the "official" way.

  correct.  when i am testing/writing code i am in the
source code directory above called frog which
looks like:

=====
-rw-------  1 me me  8962 Dec 21 00:20 active.py
-rw-------  1 me me  6673 Dec 21 00:11 background.py
-rw-------  1 me me 11796 Dec 21 00:10 board.py
-rw-------  1 me me   217 Dec 20 23:16 commands.py
-rw-------  1 me me  5987 Dec 21 00:12 config.py
-rw-------  1 me me 34077 Dec 21 00:14 dialog.py
drwx------  2 me me  4096 Dec 18 09:26 doc
-rw-------  1 me me  1943 Dec 21 00:15 history.py
-rw-------  1 me me   310 Dec 20 23:39 __init__.py
-rw-------  1 me me  1845 Dec 21 00:15 labels.py
-rw-------  1 me me   140 Dec 20 22:39 __main__.py
-rw-------  1 me me 23973 Dec 21 00:17 marbles.py
-rw-------  1 me me 13534 Dec 21 00:34 my_init.py
-rw-------  1 me me 10354 Dec 21 09:02 ngfp.py
drwx------ 10 me me  4096 Dec 18 09:26 png
-rw-------  1 me me  5514 Dec 21 00:18 randboard.py
(env) me at ant(18)~/src/salsa/frog/frog$ 
=====

  so to test the code i simply run it via

$ python3 ngfp.py

  which works as i expect it.

=====


> I work around this by using "setuptools" and its "setup"
> function in my "setup.py". "setuptools" supports the
> command "develop" (activated via "python setup.py develop")
> which "installs" the package in a way that it refers to
> the developped sources. This way, I can always
> import in the "official" way and nevertheless all
> my changes are effective immediately.

  i'm not sure how this works, but i will look into it.


> Another approach would be to lay out your package in such
> a way that it refects the installed structure and
> for testing put your script in the directory containing
> your top level packages (or extend "sys.path" with this directory).

  i think that is what i've always assumed that was
how the code is installed is exactly as how i have it
in my development directory as shown above.  your hint
here says that it probably isn't so...  heh...

  thanks, these are things a newbie doesn't know, but
someone experienced just considers common sense.

  for me coming from C code, packed and unpacked via
tarballs and such exactly where i want them to go is
the normal and it seems so strange to find things 
so hard to figure out.

  the other problem i have is that things are cached
and need to be cleared out sometimes before changes are
reflected.  i've run into this multiple times and it's
really yet another frustrating thing.

  i wish there was a way for me to tell python to never
cache anything (while i am developing) anywhere (even
during the setup.py dist/sdist/wheel stuff or installs
via pip).

  is the following missing anything i don't know about?

=====
$ rm -rf build/ dist frog/__pycache__/
$ python3 setup.py sdist bdist_wheel
=====

  thank you for the hints, i'll go read up on them more
now...  :)


  ant


From lepto.python at gmail.com  Fri Dec 21 10:34:03 2018
From: lepto.python at gmail.com (oyster)
Date: Fri, 21 Dec 2018 23:34:03 +0800
Subject: release: a function to express the 4 steps of math calculation in
 markdown format
Message-ID: <CACW-qXVWapN95CUnVN+Ap3Zegh3O47w1UjgtWrJ56dUX2-CwYg@mail.gmail.com>

I wrote a post on
https://stackoverflow.com/questions/53833884/failed-to-show-calculation-steps-with-sympy-and-markdown

Now I wrote https://github.com/retsyo/expand_expression to answer the
post partly

I released it for it can help others.

Thanks


From as at sci.fi  Fri Dec 21 16:54:53 2018
From: as at sci.fi (Anssi Saari)
Date: Fri, 21 Dec 2018 23:54:53 +0200
Subject: building 3.7.1 from source, _ctypes and libffi troubles
References: <CADjSo4SfUXb5oa9sHbFbV5GKAWdrJeMyCftVKO4xTdLmTpd+uw@mail.gmail.com>
Message-ID: <vg3ftuqpmtu.fsf@coffee.modeemi.fi>

"Fetchinson . via Python-list" <python-list at python.org> writes:

> And as far as I know pkg-config is used by python's configure script
> so everything should be fine. I also set
> LD_LIBRARY_PATH=/home/fetch/opt/lib:/home/fetch/opt/lib64 and also
> C_INCLUDE_PATH=/home/fetch/opt/include

I looked into this a little. I found that setting C_INCLUDE_PATH as you
did disables finding libffi headers since pkg-config prints nothing with
pkg-config --cflags libffi then and it seems the headers are found with
pkg-config.

As for the libraries, detect_modules() in setup.py wants to find extra
library directories from the generated makefile so you need to make sure
they actually go in there, like this for example:

LDFLAGS=`pkg-config --libs-only-L libffi` ./configure

Then you'll have CONFIGURE_LDFLAGS and PY_LDFLAGS set in the generated
makefile and setup.py will pick up the directory from there.

Anyways, hope it helps. This worked for me on Ubuntu 16.04.5 LTS.



From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Sat Dec 22 13:26:29 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Sat, 22 Dec 2018 18:26:29 +0000
Subject: cython3: Cannot start!
Message-ID: <pvlvkm$1cu2$1@adenine.netfront.net>

Hi!
Sorry if this is OT.

I decided to give cython a try and cannot run a very simple program!

1. I am using kubuntu 18.04 and installe cython3 (not cython).

2. My program tp.pyx:

# cython: language_level=3
print("Test",2)

3. setup.py
from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("tp.pyx")
)

4. Running it:
python3 setup.py build_ext --inplace
python3 -c 'import tp'

5. output:
('Test', 2)

This is wrong for python3! It should output
Test 2

It seems that it is parsing tp.pyx as a python2 script!

I tried to change the print to python2
print "Test",2

and it recognizes the syntax and outputs
Test 2

So, how can I tell cython to use python3?

Thanks for any help/comments


From python at mrabarnett.plus.com  Sat Dec 22 14:48:29 2018
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 22 Dec 2018 19:48:29 +0000
Subject: cython3: Cannot start!
In-Reply-To: <pvlvkm$1cu2$1@adenine.netfront.net>
References: <pvlvkm$1cu2$1@adenine.netfront.net>
Message-ID: <abee2bd0-7116-3d1e-40bf-3732f631651c@mrabarnett.plus.com>

On 2018-12-22 18:26, Paulo da Silva wrote:
> Hi!
> Sorry if this is OT.
> 
> I decided to give cython a try and cannot run a very simple program!
> 
> 1. I am using kubuntu 18.04 and installe cython3 (not cython).
> 
> 2. My program tp.pyx:
> 
> # cython: language_level=3
> print("Test",2)
> 
> 3. setup.py
> from distutils.core import setup
> from Cython.Build import cythonize
> 
> setup(
>      ext_modules = cythonize("tp.pyx")
> )
> 
> 4. Running it:
> python3 setup.py build_ext --inplace
> python3 -c 'import tp'
> 
> 5. output:
> ('Test', 2)
> 
> This is wrong for python3! It should output
> Test 2
> 
> It seems that it is parsing tp.pyx as a python2 script!
> 
> I tried to change the print to python2
> print "Test",2
> 
> and it recognizes the syntax and outputs
> Test 2
> 
> So, how can I tell cython to use python3?
> 
> Thanks for any help/comments
> 
Well, I've just tried this on Raspbian with the same files (for Python 3):

python3 -m pip install cython
python3 setup.py build_ext --inplace
python3 -c 'import tp'

and it printed:

Test 2


From rootsfinder at gmail.com  Sat Dec 22 16:48:24 2018
From: rootsfinder at gmail.com (rootsfinder at gmail.com)
Date: Sat, 22 Dec 2018 13:48:24 -0800 (PST)
Subject: Need help with a message embedded in an image
Message-ID: <8f0d91c5-7199-4cd8-8189-b0318f851799@googlegroups.com>

I am doing a series of puzzles.
One of the puzzles has to do with a message embedded in an image.

I am not familiar with python can someone please help me.

Here is a link to the images and the code.

https://drive.google.com/open?id=1dg5jMFtkUvKzhxv_Fqe7QMI_5x-p_CIr

 Thank you very much.


From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Sat Dec 22 18:55:23 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Sat, 22 Dec 2018 23:55:23 +0000
Subject: cython3: Cannot start! [RESOLVED]
References: <pvlvkm$1cu2$1@adenine.netfront.net>
 <abee2bd0-7116-3d1e-40bf-3732f631651c@mrabarnett.plus.com>
 <mailman.54.1545508299.31622.python-list@python.org>
Message-ID: <pvmita$34l$1@gioia.aioe.org>

?s 19:48 de 22/12/18, MRAB escreveu:
> On 2018-12-22 18:26, Paulo da Silva wrote:
...


> Well, I've just tried this on Raspbian with the same files (for Python 3):
> 
> python3 -m pip install cython
> python3 setup.py build_ext --inplace
> python3 -c 'import tp'
> 
> and it printed:
> 
> Test 2

OK. I uninstalled the distributed cython3 and installed it using pip3.
Now it works!

Thanks


From arj.python at gmail.com  Sun Dec 23 01:17:23 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sun, 23 Dec 2018 10:17:23 +0400
Subject: random choice from dictionary
Message-ID: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>

greetings,

just a check, is this:

random.choice(list(data.keys()))

the only way to get a random key from a dictionary? if not any plan to a
more convenient naming?

yours,

-- 
Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Garanti
sans virus. www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>


From rosuav at gmail.com  Sun Dec 23 03:37:16 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 23 Dec 2018 19:37:16 +1100
Subject: random choice from dictionary
In-Reply-To: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
References: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
Message-ID: <CAPTjJmoSVYjOWrPzW8o1MQtSVyQD_vTrPtbHKnTG7dXmJ3rbNw@mail.gmail.com>

On Sun, Dec 23, 2018 at 7:34 PM Abdur-Rahmaan Janhangeer
<arj.python at gmail.com> wrote:
>
> greetings,
>
> just a check, is this:
>
> random.choice(list(data.keys()))
>
> the only way to get a random key from a dictionary? if not any plan to a
> more convenient naming?

Does it have to be truly random, or are you okay with it being
arbitrary selected? Dictionaries have a popitem method that may be of
value. Otherwise, I would say what you have is fine, save that I'd
just write random.choice(list(data)) instead of using the unnecessary
keys() call.

ChrisA


From arj.python at gmail.com  Sun Dec 23 04:04:15 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sun, 23 Dec 2018 13:04:15 +0400
Subject: random choice from dictionary
In-Reply-To: <CAPTjJmoSVYjOWrPzW8o1MQtSVyQD_vTrPtbHKnTG7dXmJ3rbNw@mail.gmail.com>
References: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
 <CAPTjJmoSVYjOWrPzW8o1MQtSVyQD_vTrPtbHKnTG7dXmJ3rbNw@mail.gmail.com>
Message-ID: <CADrxXXm_WGzei+HEyku9aS8LtOXARwM6964Z4cQW3-SF5UxSJA@mail.gmail.com>

random.choice(list(data)) is great. fine with arbitrary selection.

else, given data.keys(), how do i get the first key?

yours
-- 
Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Garanti
sans virus. www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>


From bintacomputers at gmail.com  Sun Dec 23 08:29:11 2018
From: bintacomputers at gmail.com (Umar Yusuf)
Date: Sun, 23 Dec 2018 05:29:11 -0800 (PST)
Subject: Mask two images with python
In-Reply-To: <c835f128-fa2c-43be-8be9-a3b3baccc749@googlegroups.com>
References: <c835f128-fa2c-43be-8be9-a3b3baccc749@googlegroups.com>
Message-ID: <d966b8bc-41af-49bb-acc2-ea994fd902fe@googlegroups.com>

On Wednesday, 19 December 2018 06:36:01 UTC+1, Umar Yusuf  wrote:
> Hello there,
> How do I supper impose an image design on a transparent png image?
> 
> I have tried to use OpenCV's "cv2.bitwise_and" function to no success. I posted the detail question here: https://stackoverflow.com/questions/53791510/python-opencv-mask-and-glow
> 
> Thank you for your time.



Sorry for my bad explanation... I just don't know how to put this in image processing terms.

Anyway, I decided to give pillow module a look and its paste() method came close to what I wanted. So, have re-explain and posted it as new question here: https://stackoverflow.com/questions/53903607/overlay-pattern-image-on-exact-portion-of-png-image-with-python


Thank you




From ant at anthive.com  Sun Dec 23 12:46:06 2018
From: ant at anthive.com (ant)
Date: Sun, 23 Dec 2018 12:46:06 -0500
Subject: [SOLVED] Re: ah, progress...
References: <n5ooef-a7a.ln1@anthive.com> <87a7l2gg1i.fsf@handshake.de>
 <mailman.25.1545202642.31622.python-list@python.org>
 <41vpef-2n2.ln1@anthive.com> <901qef-f53.ln1@anthive.com>
 <0bbqef-u25.ln1@anthive.com> <871s6cwuog.fsf@handshake.de>
 <mailman.30.1545289149.31622.python-list@python.org>
 <0gcuef-sfc.ln1@anthive.com> <87h8f7v06v.fsf@handshake.de>
 <mailman.44.1545375314.31622.python-list@python.org>
Message-ID: <u7u4ff-jl6.ln1@anthive.com>

dieter wrote:

...

  thank you for your help.  :)  i finally worked through
the changes needed at last.


  my current package in testing PyPI is at:

  https://test.pypi.org/project/ngfp/

  which uses my code from:

  https://salsa.debian.org/ant-guest/gfpoken-in-python


  i ended up needing to do two things.  to get all the import
statements changed to include the package/module name and to
change all my image load statements to include the full path
as determined at runtime.

  this may not work on a non-linux or non-posix system from
the binary distribution but the source code distribution may
work ok.  if anyone wants to try it out.

  feedback is appreciated in any case.  :)


  now the next fun steps (figuring out the windows, other 
versions and then the Debian packaging for it).  since i
don't have windows or other machines to test on i don't 
know how that will go.

  cheers and thanks again,


  ant


From sntshkmr60 at gmail.com  Sun Dec 23 14:20:17 2018
From: sntshkmr60 at gmail.com (sntshkmr60 at gmail.com)
Date: Sun, 23 Dec 2018 11:20:17 -0800 (PST)
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
Message-ID: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>

I'm trying to upload my package to PyPi, but before that I wanted to upload my package to TestPyPi.

I'm following https://packaging.python.org/guides/using-testpypi/

I'm issuing this command: `twine upload --repository-url https://test.pypi.org/legacy/ dist/*`

Here is the output:

```
Enter your username: sntshkmr60
Enter your password:
Uploading distributions to https://test.pypi.org/
Uploading mysecretpackage-0.0.1a0-py3-none-any.whl
100%|????????????????????????????????????????????????????????????????????????| 16.7k/16.7k [00:06<00:00, 2.50kB/s]
NOTE: Try --verbose to see response content.
HTTPError: 405 Client Error: Method Not Allowed for url: https://test.pypi.org/
```

Here is the extra output from --verbose flag:

```
Content received from server:
<html>
 <head>
  <title>405 Method Not Allowed</title>
 </head>
 <body>
  <h1>405 Method Not Allowed</h1>
  The method POST is not allowed for this resource. <br/><br/>

 </body>
</html>
HTTPError: 405 Client Error: Method Not Allowed for url: https://test.pypi.org/
```

What am I doing wrong?


From avigross at verizon.net  Sun Dec 23 14:52:17 2018
From: avigross at verizon.net (Avi Gross)
Date: Sun, 23 Dec 2018 14:52:17 -0500
Subject: random choice from dictionary
In-Reply-To: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
References: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
Message-ID: <008f01d49af9$02a24470$07e6cd50$@verizon.net>

There are quite a few places the new pythonic way of doing things requires
extra steps to get an iterator to expand into a list so Abdul-Rahmann
probably is right that there is no easy way to get a random key from a
standard dictionary. Other than the expected answers to make a customized
dictionary or function that provides the functionality at the expense for
potentially generating a huge list of keys in memory, I offer the following
as an alternative for large dictionaries and especially if they have large
keys.

The following is a function that iterates over the dictionary one key at a
time and when it reaches a random one, it returns the key without further
evaluation. On the average, it takes N/2 iterations for N keys. Asking to
make a list(data) may be efficient in terms of time and indexing is fast but
space is used maximally unless it just points to existing storage.

Here is one way to do the function. Some might use an emum instead.

def rand_key(data):
    """Get a random key from a dict without using all of memory."""
    import random        
    randy = random.randrange(len(data))
    this = 0
    for key in data:
        if (this == randy):
            return(key)
        this += 1

Here is a sample showing it running:

>>> import string
		   
>>> data= { key : value for (key, value) in zip(string.ascii_uppercase,
string.ascii_lowercase) }
		   
>>> data
		   
{'A': 'a', 'B': 'b', 'C': 'c', 'D': 'd', 'E': 'e', 'F': 'f', 'G': 'g', 'H':
'h', 'I': 'i', 'J': 'j', 'K': 'k', 'L': 'l', 'M': 'm', 'N': 'n', 'O': 'o',
'P': 'p', 'Q': 'q', 'R': 'r', 'S': 's', 'T': 't', 'U': 'u', 'V': 'v', 'W':
'w', 'X': 'x', 'Y': 'y', 'Z': 'z'}
>>> rand_key(data)
		   
'X'
>>> rand_key(data)
		   
'P'
>>> rand_key(data)
		   
'D'
>>> rand_key(data)
		   
'G'
>>> rand_key(data)
		   
'H'
>>> data[rand_key(data)]
		   
'x'
>>> data[rand_key(data)]
		   
'w'
>>> data[rand_key(data)]
		   
'n'
>>> data[rand_key(data)]
		   
'h'

Of course it depends on what you want to do. If getting random keys
repeatedly, it would be best to do things like make another dictionary with
the new key being integers from 0 to the length and the new values to be the
old keys. You can then look up a random index to get a random key.

>>> dataindex = { index: key for (index, key) in enumerate(data)}
		   
>>> dataindex
		   
{0: 'A', 1: 'B', 2: 'C', 3: 'D', 4: 'E', 5: 'F', 6: 'G', 7: 'H', 8: 'I', 9:
'J', 10: 'K', 11: 'L', 12: 'M', 13: 'N', 14: 'O', 15: 'P', 16: 'Q', 17: 'R',
18: 'S', 19: 'T', 20: 'U', 21: 'V', 22: 'W', 23: 'X', 24: 'Y', 25: 'Z'}
>>> dataindex[11]
		   
'L'
>>> import random
		   
>>> dataindex[random.randrange(len(dataindex))]
		   
'Y'
>>> dataindex[random.randrange(len(dataindex))]
		   
'O'

Again, depending on your needs, some solutions may be worth extra code, some
may not.


-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Abdur-Rahmaan Janhangeer
Sent: Sunday, December 23, 2018 1:17 AM
To: Python <python-list at python.org>
Subject: random choice from dictionary

greetings,

just a check, is this:

random.choice(list(data.keys()))

the only way to get a random key from a dictionary? if not any plan to a
more convenient naming?

yours,

--
Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ Mauritius

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai
gn=sig-email&utm_content=webmail>
Garanti
sans virus. www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai
gn=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
--
https://mail.python.org/mailman/listinfo/python-list



From ant at anthive.com  Sun Dec 23 14:57:56 2018
From: ant at anthive.com (ant)
Date: Sun, 23 Dec 2018 14:57:56 -0500
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
References: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>
Message-ID: <4v55ff-ii8.ln1@anthive.com>

sntshkmr60 at gmail.com wrote:
> I'm trying to upload my package to PyPi, but before that I wanted to upload my package to TestPyPi.
>
> I'm following https://packaging.python.org/guides/using-testpypi/
>
> I'm issuing this command: `twine upload --repository-url https://test.pypi.org/legacy/ dist/*`
>
> Here is the output:
>
> ```
> Enter your username: sntshkmr60
> Enter your password:
> Uploading distributions to https://test.pypi.org/
> Uploading mysecretpackage-0.0.1a0-py3-none-any.whl
> 100%|????????????????????????????????????????????????????????????????????????| 16.7k/16.7k [00:06<00:00, 2.50kB/s]
> NOTE: Try --verbose to see response content.
> HTTPError: 405 Client Error: Method Not Allowed for url: https://test.pypi.org/
> ```
>
> Here is the extra output from --verbose flag:
>
> ```
> Content received from server:
><html>
> <head>
>  <title>405 Method Not Allowed</title>
> </head>
> <body>
>  <h1>405 Method Not Allowed</h1>
>   The method POST is not allowed for this resource. <br/><br/>
>
> </body>
></html>
> HTTPError: 405 Client Error: Method Not Allowed for url: https://test.pypi.org/
> ```
>
> What am I doing wrong?

  well, i used it earlier so here is how i have it set up
as follows.

  because this is testing repository i am not worried about
password being stored in a file:

.pypirc 
=====
[distutils]
index-servers=
    testpypi

[testpypi]
repository: https://test.pypi.org/legacy/
username: UserName
password: Password
=====


and my upload command is:


=====
#!/bin/sh
#
# upload ngfp to test pypi

NGFP_SRC_HOME="/home/me/src/salsa/ngfp"
if test ! -d "$NGFP_SRC_HOME" ; then
  echo "  directory $NGFP_SRC_HOME does not exist!"
  exit
fi

cd $NGFP_SRC_HOME

twine upload --repository testpypi dist/*
=====


  note: it may take a while for what you uploaded to be made
available for download again even if it reflected in your
project page.  sometimes it has been as long as a half hour
or more before it comes down.  other times it has only been
a few minutes.

  hope this helps...


  ant


From ant at anthive.com  Sun Dec 23 15:49:44 2018
From: ant at anthive.com (ant)
Date: Sun, 23 Dec 2018 15:49:44 -0500
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
References: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>
 <4v55ff-ii8.ln1@anthive.com>
Message-ID: <8095ff-qb9.ln1@anthive.com>

ant wrote:

...

> .pypirc 
>=====
> [distutils]
> index-servers=
>     testpypi
>
> [testpypi]
> repository: https://test.pypi.org/legacy/
> username: UserName
> password: Password
>=====
>
>
> and my upload command is:
>
>
>=====
> #!/bin/sh
> #
> # upload ngfp to test pypi
>
> NGFP_SRC_HOME="/home/me/src/salsa/ngfp"
> if test ! -d "$NGFP_SRC_HOME" ; then
>   echo "  directory $NGFP_SRC_HOME does not exist!"
>   exit
> fi
>
> cd $NGFP_SRC_HOME
>
> twine upload --repository testpypi dist/*
>=====


  since i just went through this myself i thought i'd
add a bit more.

  when you change over to post to the main pypi.org
you need to do it to upload.pypi.org like:

=====
[pypi]
repository: https://upload.pypi.org/legacy/
=====

  as i was getting 405 errors when using just pypi.org
i suspect your error above is that you're not using the
/legacy/ part at the end...

  good luck!


  ant


From sntshkmr60 at gmail.com  Sun Dec 23 16:08:57 2018
From: sntshkmr60 at gmail.com (sntshkmr60 at gmail.com)
Date: Sun, 23 Dec 2018 13:08:57 -0800 (PST)
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
In-Reply-To: <8095ff-qb9.ln1@anthive.com>
References: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>
 <4v55ff-ii8.ln1@anthive.com> <8095ff-qb9.ln1@anthive.com>
Message-ID: <5de0e390-d62d-4e0c-820d-544fa8069cc1@googlegroups.com>

> .pypirc 
>=====
> [distutils]
> index-servers=
>     testpypi
>
> [testpypi]
> repository: https://test.pypi.org/legacy/
> username: UserName
> password: Password


> twine upload --repository testpypi dist/*

Tried your suggestion for .pypirc file. Still the same error.

Has something changed and is not reflected in the documentation?


From python at mrabarnett.plus.com  Sun Dec 23 16:21:29 2018
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 23 Dec 2018 21:21:29 +0000
Subject: random choice from dictionary
In-Reply-To: <008f01d49af9$02a24470$07e6cd50$@verizon.net>
References: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
 <008f01d49af9$02a24470$07e6cd50$@verizon.net>
Message-ID: <20190c41-da5b-480a-2ccd-ce593d0f90e0@mrabarnett.plus.com>

On 2018-12-23 19:52, Avi Gross wrote:
> There are quite a few places the new pythonic way of doing things requires
> extra steps to get an iterator to expand into a list so Abdul-Rahmann
> probably is right that there is no easy way to get a random key from a
> standard dictionary. Other than the expected answers to make a customized
> dictionary or function that provides the functionality at the expense for
> potentially generating a huge list of keys in memory, I offer the following
> as an alternative for large dictionaries and especially if they have large
> keys.
> 
> The following is a function that iterates over the dictionary one key at a
> time and when it reaches a random one, it returns the key without further
> evaluation. On the average, it takes N/2 iterations for N keys. Asking to
> make a list(data) may be efficient in terms of time and indexing is fast but
> space is used maximally unless it just points to existing storage.
> 
> Here is one way to do the function. Some might use an emum instead.
> 
> def rand_key(data):
>      """Get a random key from a dict without using all of memory."""
>      import random
>      randy = random.randrange(len(data))
>      this = 0
>      for key in data:
>          if (this == randy):
>              return(key)
>          this += 1
> 
[snip]
You might as well use 'enumerate' there:

def rand_key(data):
     """Get a random key from a dict without using all of memory."""
     import random
     randy = random.randrange(len(data))
     for this, key in enumerate(data):
         if this == randy:
             return key

Or you could use 'islice':

def rand_key(data):
     """Get a random key from a dict without using all of memory."""
     import itertools
     import random
     randy = random.randrange(len(data))
     return list(itertools.islice(data, randy, randy + 1))[0]


From ant at anthive.com  Sun Dec 23 17:36:32 2018
From: ant at anthive.com (ant)
Date: Sun, 23 Dec 2018 17:36:32 -0500
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
References: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>
 <4v55ff-ii8.ln1@anthive.com> <8095ff-qb9.ln1@anthive.com>
 <5de0e390-d62d-4e0c-820d-544fa8069cc1@googlegroups.com>
Message-ID: <g8f5ff-3hb.ln1@anthive.com>

sntshkmr60 at gmail.com wrote:
>> .pypirc 
>>=====
>> [distutils]
>> index-servers=
>>     testpypi
>>
>> [testpypi]
>> repository: https://test.pypi.org/legacy/
>> username: UserName
>> password: Password
>
>
>> twine upload --repository testpypi dist/*
>
> Tried your suggestion for .pypirc file. Still the same error.
>
> Has something changed and is not reflected in the documentation?


  did you check your dists via twine?


  $ twine check --verbose dist/*

  also make sure you are using up to date versions of
setuptools, wheel and twine


  $ pip install --upgrade setuptools
  $ pip install --upgrade wheel
  $ pip install --upgrade twine


  i just used it earlier today...  worked fine for me.
is your account there and verified?


  ant


From sntshkmr60 at gmail.com  Sun Dec 23 19:25:05 2018
From: sntshkmr60 at gmail.com (sntshkmr60 at gmail.com)
Date: Sun, 23 Dec 2018 16:25:05 -0800 (PST)
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
In-Reply-To: <g8f5ff-3hb.ln1@anthive.com>
References: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>
 <4v55ff-ii8.ln1@anthive.com> <8095ff-qb9.ln1@anthive.com>
 <5de0e390-d62d-4e0c-820d-544fa8069cc1@googlegroups.com>
 <g8f5ff-3hb.ln1@anthive.com>
Message-ID: <1790af32-98aa-4c12-9ef0-04539798c4d0@googlegroups.com>

>   did you check your dists via twine?
> 
>   $ twine check --verbose dist/*

I checked with `twine check dist/*` (there's no --verbose option though).
I only get warning about markdown

>   also make sure you are using up to date versions of
> setuptools, wheel and twine
> 
> 
>   $ pip install --upgrade setuptools
>   $ pip install --upgrade wheel
>   $ pip install --upgrade twine

[65]> import setuptools, wheel, twine
[66]> print(setuptools.__version__)
40.6.3
[67]> print(wheel.__version__)
0.32.3
[68]> print(twine.__version__)
1.12.1


>   i just used it earlier today...  worked fine for me.
> is your account there and verified?

Yes, I have an account on TestPyPI and is verified. 



From ant at anthive.com  Sun Dec 23 19:59:44 2018
From: ant at anthive.com (ant)
Date: Sun, 23 Dec 2018 19:59:44 -0500
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
References: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>
 <4v55ff-ii8.ln1@anthive.com> <8095ff-qb9.ln1@anthive.com>
 <5de0e390-d62d-4e0c-820d-544fa8069cc1@googlegroups.com>
 <g8f5ff-3hb.ln1@anthive.com>
 <1790af32-98aa-4c12-9ef0-04539798c4d0@googlegroups.com>
Message-ID: <0ln5ff-g1e.ln1@anthive.com>

sntshkmr60 at gmail.com wrote:
>>   did you check your dists via twine?
>> 
>>   $ twine check --verbose dist/*
>
> I checked with `twine check dist/*` (there's no --verbose option though).
> I only get warning about markdown

  right, i see later in my history i used the twine
check without the verbose option.

  you have to fix that for it to upload...

  $ pip install readme_renderer[md]


>>   also make sure you are using up to date versions of
>> setuptools, wheel and twine
>> 
>> 
>>   $ pip install --upgrade setuptools
>>   $ pip install --upgrade wheel
>>   $ pip install --upgrade twine
>
> [65]> import setuptools, wheel, twine
> [66]> print(setuptools.__version__)
> 40.6.3
> [67]> print(wheel.__version__)
> 0.32.3
> [68]> print(twine.__version__)
> 1.12.1
>
>
>>   i just used it earlier today...  worked fine for me.
>> is your account there and verified?
>
> Yes, I have an account on TestPyPI and is verified. 

  try the above...


  ant


From ben.usenet at bsb.me.uk  Sun Dec 23 22:09:14 2018
From: ben.usenet at bsb.me.uk (Ben Bacarisse)
Date: Mon, 24 Dec 2018 03:09:14 +0000
Subject: random choice from dictionary
References: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
 <008f01d49af9$02a24470$07e6cd50$@verizon.net>
 <mailman.0.1545594745.2779.python-list@python.org>
Message-ID: <875zvjhb8l.fsf@bsb.me.uk>

"Avi Gross" <avigross at verizon.net> writes:
<snip>
> The following is a function that iterates over the dictionary one key at a
> time and when it reaches a random one, it returns the key without further
> evaluation. On the average, it takes N/2 iterations for N keys. Asking to
> make a list(data) may be efficient in terms of time and indexing is fast but
> space is used maximally unless it just points to existing storage.
>
> Here is one way to do the function. Some might use an emum instead.
>
> def rand_key(data):
>     """Get a random key from a dict without using all of memory."""
>     import random        
>     randy = random.randrange(len(data))
>     this = 0
>     for key in data:
>         if (this == randy):
>             return(key)
>         this += 1

There is beautiful variation on this that can select k random keys:

def sample_keys(data, k = 1):
    """Return k random keys from a dict."""
    import random
    items = len(data)
    keys = []
    for key in data:
        r = random.random()
        if r < k/items:
            keys.append(key)
            k -= 1
        items -= 1
    return keys

And there is a variation on /that/ will use fewer random numbers --
essentially picking a number of "skip" distances as you pick one.  But
it's late, and I need to get ready to eat lots...

<snip>
-- 
Ben.


From sarah123ed at gmail.com  Mon Dec 24 00:12:07 2018
From: sarah123ed at gmail.com (sarah123ed at gmail.com)
Date: Sun, 23 Dec 2018 21:12:07 -0800 (PST)
Subject: random choice from dictionary
In-Reply-To: <mailman.55.1545553981.31622.python-list@python.org>
References: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
 <mailman.55.1545553981.31622.python-list@python.org>
Message-ID: <1ab31887-f233-41ee-aa07-0c95c5bb5433@googlegroups.com>


Maybe something like this?

    import random
    k = list(data.keys())
    random.shuffle(k)
    selected = f[k[0]]


From sarah123ed at gmail.com  Mon Dec 24 00:17:02 2018
From: sarah123ed at gmail.com (sarah123ed at gmail.com)
Date: Sun, 23 Dec 2018 21:17:02 -0800 (PST)
Subject: random choice from dictionary
In-Reply-To: <mailman.55.1545553981.31622.python-list@python.org>
References: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
 <mailman.55.1545553981.31622.python-list@python.org>
Message-ID: <3a1569d3-1a94-445b-a63b-31fcfc662504@googlegroups.com>



Maybe something like this:

    import random
    k = list(data.keys())
    random.shuffle(k)
    result = data[k[0]]


    


From avigross at verizon.net  Mon Dec 24 00:53:17 2018
From: avigross at verizon.net (Avi Gross)
Date: Mon, 24 Dec 2018 00:53:17 -0500
Subject: random choice from dictionary
In-Reply-To: <20190c41-da5b-480a-2ccd-ce593d0f90e0@mrabarnett.plus.com>
References: <CADrxXXnwGm6DUW1EDRpnsiT5=mQ5fmG0AnjzbM8pYyTUgCpsAA@mail.gmail.com>
 <008f01d49af9$02a24470$07e6cd50$@verizon.net>
 <20190c41-da5b-480a-2ccd-ce593d0f90e0@mrabarnett.plus.com>
Message-ID: <002c01d49b4c$f7cb42a0$e761c7e0$@verizon.net>

I did say some would use enumerate but I was rushing to go to my nephews wedding and my enum became emum which is meaningless ?

If the goal is to make a relatively efficient algorithm and the index made by enumerate is not needed, then I judged that the overhead of enumerate(data) returning two values to be unpacked each time through the loop might be more than having a single counter. Of course enumerate likely is considered more pythonic ?

Similarly, the suggestion to use islice (although the name is also unfortunately reminiscent of is lice) may be a good one. But it basically does what my function did, perhaps as fast and perhaps not. My impression is it iterates using next() a number of times then stops. It is so simple, there might be no need for my function. The request was to find an nth item and you can feed (the random) n directly into islice.

Just for amusement, I already get a random word a day in my email from two dictionary sources in English as well as quite a few in other languages. I not only get the word as a key, but also a translation and even an example sentence with translation.

But I doubt the words are chosen randomly and they are not likely to be from a python dictionary.?

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of MRAB
Sent: Sunday, December 23, 2018 4:21 PM
To: python-list at python.org
Subject: Re: random choice from dictionary

On 2018-12-23 19:52, Avi Gross wrote:
> There are quite a few places the new pythonic way of doing things 
> requires extra steps to get an iterator to expand into a list so 
> Abdul-Rahmann probably is right that there is no easy way to get a 
> random key from a standard dictionary. Other than the expected answers 
> to make a customized dictionary or function that provides the 
> functionality at the expense for potentially generating a huge list of 
> keys in memory, I offer the following as an alternative for large 
> dictionaries and especially if they have large keys.
> 
> The following is a function that iterates over the dictionary one key 
> at a time and when it reaches a random one, it returns the key without 
> further evaluation. On the average, it takes N/2 iterations for N 
> keys. Asking to make a list(data) may be efficient in terms of time 
> and indexing is fast but space is used maximally unless it just points to existing storage.
> 
> Here is one way to do the function. Some might use an emum instead.
> 
> def rand_key(data):
>      """Get a random key from a dict without using all of memory."""
>      import random
>      randy = random.randrange(len(data))
>      this = 0
>      for key in data:
>          if (this == randy):
>              return(key)
>          this += 1
> 
[snip]
You might as well use 'enumerate' there:

def rand_key(data):
     """Get a random key from a dict without using all of memory."""
     import random
     randy = random.randrange(len(data))
     for this, key in enumerate(data):
         if this == randy:
             return key

Or you could use 'islice':

def rand_key(data):
     """Get a random key from a dict without using all of memory."""
     import itertools
     import random
     randy = random.randrange(len(data))
     return list(itertools.islice(data, randy, randy + 1))[0]
--
https://mail.python.org/mailman/listinfo/python-list



From nad at python.org  Mon Dec 24 06:57:53 2018
From: nad at python.org (Ned Deily)
Date: Mon, 24 Dec 2018 06:57:53 -0500
Subject: [RELEASE] Python 3.7.2 and 3.6.8 are now available
Message-ID: <3041C039-C5F6-4A01-BEB8-783238B07970@python.org>

https://blog.python.org/2018/12/python-372-and-368-are-now-available.html

Python 3.7.2 and 3.6.8 are now available. Python 3.7.2 is the next
maintenance release of Python 3.7, the latest feature release of Python.
You can find Python 3.7.2 here:
    https://www.python.org/downloads/release/python-372/

See the What?s New In Python 3.7 document for more information about the
many new features and optimizations included in the 3.7 series. Detailed
information about the changes made in 3.7.2 can be found in its change log.

We are also happy to announce the availability of Python 3.6.8:
    https://www.python.org/downloads/release/python-368/

Python 3.6.8 is planned to be the last bugfix release of Python 3.6. Per
our support policy, we plan to provide security fixes for Python 3.6 as
needed through 2021, five years following its initial release.

Thanks to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python
Software Foundation.

https://docs.python.org/3.7/whatsnew/3.7.html
https://docs.python.org/3.7/whatsnew/changelog.html#python-3-7-2-final
https://docs.python.org/3.6/whatsnew/changelog.html#python-3-6-8-final
https://www.python.org/psf/

--
  Ned Deily
  nad at python.org -- []



From stefan_ml at behnel.de  Mon Dec 24 09:07:07 2018
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 24 Dec 2018 15:07:07 +0100
Subject: cython3: Cannot start!
In-Reply-To: <pvlvkm$1cu2$1@adenine.netfront.net>
References: <pvlvkm$1cu2$1@adenine.netfront.net>
Message-ID: <pvqp28$5uq$1@blaine.gmane.org>

Paulo da Silva schrieb am 22.12.18 um 19:26:
> Sorry if this is OT.
> 
> I decided to give cython a try and cannot run a very simple program!
> 
> 1. I am using kubuntu 18.04 and installe cython3 (not cython).
> 
> 2. My program tp.pyx:
> 
> # cython: language_level=3
> print("Test",2)
> 
> 3. setup.py
> from distutils.core import setup
> from Cython.Build import cythonize
> 
> setup(
>     ext_modules = cythonize("tp.pyx")
> )
> 
> 4. Running it:
> python3 setup.py build_ext --inplace
> python3 -c 'import tp'
> 
> 5. output:
> ('Test', 2)
> 
> This is wrong for python3! It should output
> Test 2
> 
> It seems that it is parsing tp.pyx as a python2 script!
> 
> I tried to change the print to python2
> print "Test",2
> 
> and it recognizes the syntax and outputs
> Test 2
> 
> So, how can I tell cython to use python3?

Ubuntu 18.04 ships Cython 0.26, which has a funny bug that you hit above.
It switches the language-level too late, so that the first token (or word)
in the file is parsed with Py2 syntax. In your case, that's the print
statement, which is really parsed as (Py2) statement here, not as (Py3)
function. In normal cases, the language level does not matter for the first
statement in the source (because, why would you have a print() there?), so
it took us a while to find this bug.

pip-installing Cython will get you the latest release, where this bug is
resolved.

Stefan



From stefan_ml at behnel.de  Mon Dec 24 09:24:11 2018
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 24 Dec 2018 15:24:11 +0100
Subject: Side by side comparison - CPython, nuitka, PyPy
In-Reply-To: <5f2ca83d-b1c0-facf-8b92-c5f9805e74c8@btinternet.com>
References: <5f2ca83d-b1c0-facf-8b92-c5f9805e74c8@btinternet.com>
Message-ID: <pvqq27$fcu$1@blaine.gmane.org>

Anthony Flury via Python-list schrieb am 21.12.18 um 09:06:
> I thought I would look at a side by side comparison of CPython, nuitka and
> PyPy

Interesting choice. Why nuitka?


> *The functionality under test*
> 
> I have a library (called primelib) which implements a Sieve of Erathoneses
> in pure Python - it was orginally written as part of my project Euler attempts
> 
> Not only does it build a sieve to test primality, it also builds an
> iterable list of primes, and has functionality to calculate the prime
> factors (and exponents) and also calculate all divisors of a given integer
> (up to the size of the sieve).
> 
> To test the primelib there is a simple harness which :
> 
> ?* Builds a sieve for integers from 2 to 104729 (104729 is the 10,000th
> ?? prime number)
> ?* Using a pre-built list from primes.utm.edu -
> ???? o For every integer from 2 to 104729 the prime sieve and pre-built
> ?????? list agree on the primality or non-primality
> ???? o confirm that the list of ALL primes identified by the sieve is
> ?????? the same as the pre-built list.
> ???? o For every integer from 2 to 104729, get primelib to generate the
> ?????? prime factors and exponents - and comfirm that they multiply up
> ?????? to the expected integer
> ???? o For every integer from 2 to 104729 get primelib to generate the
> ?????? divisors on the integer, and confirm that each divisor does
> ?????? divide cleanly into the integer
> 
> The Sieve is rebuilt between each test, there is no caching of data between
> test cases, so the test harness forces a lot of recalculations.
> 
> I have yet to convert primelib to be Python 3 compatible.
> 
> Exactly the same test harness was run in all 3 cases :
> 
> ?* Under CPython 2.7.15, the execution of the test harness took around
> ?? 75 seconds to execute over 5 runs - fastest 73, slowest 78.
> ?* Under Nuitka 0.6, the execution of the test harness after compiler
> ?? took around 85 seconds over 5 runes, fastest 84, slowest 86.
> ?* Under PyPy, the execution of the test harness took 4.9 seconds on
> ?? average over 5 runs, fastest 4.79, slowest 5.2
> 
> I was very impressed at the execution time improvement under PyPy, and a
> little surprised about the lack of improvement under Nuitka.
> 
> I know Nuitka is a work in progress, but given that Nuitka compiles Python
> to C code I would have expected some level of gain, especially in a maths
> heavy implementation.

It compiles to C, yes, but that by itself doesn't mean that it makes it run
faster. Remember that CPython is also written in C, so why should a simple
static translation from Python code to C make it run faster than in CPython?

Cython [1], on the other hand, is an optimising Python-to-C compiler, which
aims to generate fast code and allow users to manually tune it. That's when
you start getting real speedups that are relevant for real-world code.


> This comparison is provided for information only, and is not intended as
> any form of formal benchmark. I don't claim that primelib is as efficient
> as it could be - although every effort was made to try to make it as fast
> as I could.

I understand that it came to life as an exercise, and you probably won't
make production use of it. Actually, I doubt that there is a shortage of
prime detection libraries. ;) Still, thanks for the writeup. It's helpful
to see comparisons of "code how people write it" under different runtimes
from time to time.

Stefan


[1] http://cython.org/



From stefan_ml at behnel.de  Mon Dec 24 09:45:54 2018
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 24 Dec 2018 15:45:54 +0100
Subject: Fastest first
In-Reply-To: <006f01d4959b$8c77adb0$a5670910$@verizon.net>
References: <006f01d4959b$8c77adb0$a5670910$@verizon.net>
Message-ID: <pvqrav$c5e$1@blaine.gmane.org>

Avi Gross schrieb am 17.12.18 um 01:00:
> SHORT VERSION: a way to automatically run multiple algorithms in parallel
> and kill the rest when one returns an answer.

One (somewhat seasonal) comment on this: it doesn't always have to be about
killing (processes or threads). You might also consider a cooperative
implementation, where each of the algorithms is allowed to advance by one
"step" in each "round", and is simply discarded when a solution is found
elsewhere, or when it becomes unlikely that this specific algorithm will
contribute a future solution. This could be implemented via a sequence of
generators or coroutines in Python. Such an approach is often used in
simulations (e.g. SimPy and other "discrete event" simulators), where exact
control over the concurrency pattern is desirable.

Stefan



From sntshkmr60 at gmail.com  Mon Dec 24 10:50:37 2018
From: sntshkmr60 at gmail.com (sntshkmr60 at gmail.com)
Date: Mon, 24 Dec 2018 07:50:37 -0800 (PST)
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
In-Reply-To: <0ln5ff-g1e.ln1@anthive.com>
References: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>
 <4v55ff-ii8.ln1@anthive.com> <8095ff-qb9.ln1@anthive.com>
 <5de0e390-d62d-4e0c-820d-544fa8069cc1@googlegroups.com>
 <g8f5ff-3hb.ln1@anthive.com>
 <1790af32-98aa-4c12-9ef0-04539798c4d0@googlegroups.com>
 <0ln5ff-g1e.ln1@anthive.com>
Message-ID: <f36cccde-4e93-4d83-a3c1-0502c16d5357@googlegroups.com>

>   $ pip install readme_renderer[md]

Thanks a lot for this, I wasn't able to figure it out earlier.


From p_s_d_a_s_i_l_v_a_ns at netcabo.pt  Mon Dec 24 12:13:04 2018
From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva)
Date: Mon, 24 Dec 2018 17:13:04 +0000
Subject: cython3: Cannot start!
References: <pvlvkm$1cu2$1@adenine.netfront.net>
 <pvqp28$5uq$1@blaine.gmane.org>
 <mailman.5.1545660436.2779.python-list@python.org>
Message-ID: <pvr42v$1rkf$1@gioia.aioe.org>

?s 14:07 de 24/12/18, Stefan Behnel escreveu:
> Paulo da Silva schrieb am 22.12.18 um 19:26:
...

> 
> Ubuntu 18.04 ships Cython 0.26, which has a funny bug that you hit above.
> It switches the language-level too late, so that the first token (or word)
> in the file is parsed with Py2 syntax. In your case, that's the print
> statement, which is really parsed as (Py2) statement here, not as (Py3)
> function. In normal cases, the language level does not matter for the first
> statement in the source (because, why would you have a print() there?), so
> it took us a while to find this bug.
> 
> pip-installing Cython will get you the latest release, where this bug is
> resolved.
> 

Thank you Stefan for the clarification.
Regards.
Paulo



From ant at anthive.com  Mon Dec 24 15:37:08 2018
From: ant at anthive.com (ant)
Date: Mon, 24 Dec 2018 15:37:08 -0500
Subject: Why am I getting Error 405 while uploading my package to
 https://test.pypi.org/legacy?
References: <d76f4771-681d-47b2-b00d-2b32016ad3c5@googlegroups.com>
 <4v55ff-ii8.ln1@anthive.com> <8095ff-qb9.ln1@anthive.com>
 <5de0e390-d62d-4e0c-820d-544fa8069cc1@googlegroups.com>
 <g8f5ff-3hb.ln1@anthive.com>
 <1790af32-98aa-4c12-9ef0-04539798c4d0@googlegroups.com>
 <0ln5ff-g1e.ln1@anthive.com>
 <f36cccde-4e93-4d83-a3c1-0502c16d5357@googlegroups.com>
Message-ID: <kks7ff-jb4.ln1@anthive.com>

sntshkmr60 at gmail.com wrote:
>>   $ pip install readme_renderer[md]
>
> Thanks a lot for this, I wasn't able to figure it out earlier.

  does it work ok now?  got the upload to go to pypitest?


  ant


From avigross at verizon.net  Mon Dec 24 16:29:51 2018
From: avigross at verizon.net (Avi Gross)
Date: Mon, 24 Dec 2018 16:29:51 -0500
Subject: Fastest first
In-Reply-To: <pvqrav$c5e$1@blaine.gmane.org>
References: <006f01d4959b$8c77adb0$a5670910$@verizon.net>
 <pvqrav$c5e$1@blaine.gmane.org>
Message-ID: <003801d49bcf$ce1eaf40$6a5c0dc0$@verizon.net>

Stefan,

Yes, techniques like the one you mention are some of the ways I am aware of but they may not be the easiest way to automate.

I was thinking of a style of solving problems. I note python (on various platforms) has an amazing number of ways to do things in parallel and quite a few methods those tasks can communicate with the parent(s) or each other.

My goal was not so much to kill processes but to make a killing.

OK, maybe not. Just a pun.

The obvious goal is to have a framework where you have multiple parties trying to solve a problem using divergent methods with no obvious way in advance to figure out which would be first with a good answer. Some of the algorithms used might be heuristic and you might not want the first one that finishes but rather the first one that returns a value say within 2% of the best possible result. An example would be designing an airplane that comes within 2% of the results of one with negligible friction flying in fairly empty space. If the best current plane only is within 20% of that then any algorithm getting within 10% might be good enough.

And some algorithms may not reliably terminate on the data supplied. Eventually they need to be stopped even if no other solution comes in.

Your message seems to be talking about threads running within the same process as the parent and being time switched. If writing code for that environment, I am aware of many ways of building the module as described. Yes, you want the threads to regularly yield back control for many reasons including some dominating at the expense of others. So if they are regularly pausing and yielding, you can set it up so they get a message and perform carefully controlled apoptosis. They might carefully clean up what is needed, such as closing open files, release any locks, delete any objects exclusively used by them, perhaps scribble away some data, and exit. Sure, the parent could just kill them but that can leave a mess.

And your other suggestions also make sense in some scenarios. Yes, a thread that seems to be making slow progress might even suspend itself for a while and check to see if others have made more progress.

Let me be very concrete here. I am thinking of an example such as wanting to calculate pi to some enormous number of decimal places. Don't ask why or it will spoil the mood.

There are a stupendous number of ways to calculate pi. Many involve summing the results of an infinite series. Some do very little on each iteration and some do very complex things. Most of them can be viewed as gradually having more and more digits trailing off to the "right" that no longer change with each iteration while the ones further to the right are clearly not significant yet since they keep changing. So an algorithm might stop every hundred iterations and calculate how many digits seem quite firm since the last check. One algorithm might report to a central bulleting board that they are getting five more digits and another might report 0 and yet another might report 352. It might make sense for only the fastest ones to continue at full pace. But if an algorithm later slows down, perhaps another one should be revived. A quick search online shows lots of ways to do such a set of analyses but note many do not return pi directly but something like pi/4 or 1/pi. There is even a formula that calculates the nth digit of pi without calculating any that came before!

Clearly you might be able to start hundreds of such methods at once. It may be better not to have them all within one process. Since none of them would terminate before reaching say the billionth digit, and even the fastest may run for a very long time, you can imagine an algorithm where each one checks their progress periodically and the slowest one quite every 10 minutes till only the fastest survive and perhaps are now running faster.

But none of this applies to my original question. I am talking about whether anyone has created a general framework for a much simpler scenario. Loosely speaking, I might be asked to create a list of objects or functions that somehow can be started independently in the same way. I hand this over to some module that takes my list and does the rest and eventually returns results. That module might accept additional choices and instructions that fine tune things. But the overall code might look like:

Import some module
Make a new object that will run things for me as some kind of controller.
Invoke incantations on that controller that include setting things and especially getting it the list of things to do.
I ask it to get started and either wait or go do something else.
At some later point I find it has completed. The results of the problem are somewhere I can reach. Perhaps there are logs I can search to see more details like which methods quit or were stopped or what their last result was before ...

Please see the above as an outline or example, not exact details or designs.

I simply think such a utility may already have been built out there, as well as many variations. Just as an example, someone recently mentioned to me that there is a module with a function called islice() that might do exactly what was wanted. So, of course, I read about everything else in that module: https://docs.python.org/2/library/itertools.html

And I found quite a variety of other functions to create combinations of iterables and ways to combine them to make even more. Others have already considered what might be of use and many are being shared. And many are well debugged so unless I am trying to deliberately figure out how to do something by myself (which I admit I often do) this is a good way to go.

So although I am happy to discuss ways to do this kind of task, I am really asking if this is a common enough way of looking at solving problems that many solutions have already appeared including some that are out there and ready.

Clearly though, some such solutions would be quite specific to particular methods of dividing up tasks such as using processes versus threads or even distributing computation across the cloud. Some might be even more dynamic and allow some of the processes running to throw additional ones into the mix as variants of their methods.

I will stop here even if it means I have to kill myself. ?

Avi
-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of Stefan Behnel
Sent: Monday, December 24, 2018 9:46 AM
To: python-list at python.org
Subject: Re: Fastest first

Avi Gross schrieb am 17.12.18 um 01:00:
> SHORT VERSION: a way to automatically run multiple algorithms in 
> parallel and kill the rest when one returns an answer.

One (somewhat seasonal) comment on this: it doesn't always have to be about killing (processes or threads). You might also consider a cooperative implementation, where each of the algorithms is allowed to advance by one "step" in each "round", and is simply discarded when a solution is found elsewhere, or when it becomes unlikely that this specific algorithm will contribute a future solution. This could be implemented via a sequence of generators or coroutines in Python. Such an approach is often used in simulations (e.g. SimPy and other "discrete event" simulators), where exact control over the concurrency pattern is desirable.

Stefan

--
https://mail.python.org/mailman/listinfo/python-list



From cs at cskk.id.au  Mon Dec 24 19:33:13 2018
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 25 Dec 2018 11:33:13 +1100
Subject: Fastest first
In-Reply-To: <003801d49bcf$ce1eaf40$6a5c0dc0$@verizon.net>
References: <003801d49bcf$ce1eaf40$6a5c0dc0$@verizon.net>
Message-ID: <20181225003313.GA79784@cskk.homeip.net>

On 24Dec2018 16:29, Avi Gross <avigross at verizon.net> wrote:
>But none of this applies to my original question. I am talking about 
>whether anyone has created a general framework for a much simpler 
>scenario. Loosely speaking, I might be asked to create a list of 
>objects or functions that somehow can be started independently in the 
>same way. I hand this over to some module that takes my list and does 
>the rest and eventually returns results. That module might accept 
>additional choices and instructions that fine tune things. But the 
>overall code might look like:
>
>Import some module
>Make a new object that will run things for me as some kind of controller.
>Invoke incantations on that controller that include setting things and especially getting it the list of things to do.
>I ask it to get started and either wait or go do something else.
>At some later point I find it has completed. The results of the problem 
>are somewhere I can reach. Perhaps there are logs I can search to see 
>more details like which methods quit or were stopped or what their last 
>result was before ...

Well, I have a pair of modules named "cs.result" and "cs.later" in PyPI 
which offer this.  I imagine the futures module also does: I had 
cs.later before that arrived.

cs.result has a class called a Result, instances are objects with a 
.result attribute which may be fulfilled later; they're also callable 
which returns the .result - when it becomes ready.  cs.later is a 
capacity managed queue using these Result objects.

Basicly, you can use a cs.later.Later to submit functions to be run, or 
make cs.result.Results directly and tell them to fulfil with an 
arbitrary function. Both of these dispatch Threads to run the function.

Having submitting a bunch of functions and having the corresponding 
Results to hand, you can wait for the Results as they complete, which 
does what you're after: get the earliest result first.

Typical code looks like:

    # a Later with 4 permitted parallel tasks at a given time
    L = Later(4)
    LFs = []
    # get a LateFunction from the Later (a Result subclass)
    LF = Later.defer(function, *args, **kwargs)
    LFs.append(LF)

You can submit as many as you like, and keep them in a list for example 
such as "LFs" above.

There's a report generator function which takes a collection of Results 
and yields them as they complete.

    for LF in report(LFs):
        print(LF, "completed")

You can clearly build on this depending on your use case.

The typical implementation uses Threads, but of course a Thread can run 
a subprocess if that's sensible, and so forth, or you can use some other 
system to fulfil the Results; from the outside the user doesn't care.

Cheers,
Cameron Simpson <cs at cskk.id.au>


From uri at speedy.net  Tue Dec 25 07:45:30 2018
From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=)
Date: Tue, 25 Dec 2018 14:45:30 +0200
Subject: Why doesn't a dictionary work in classes?
Message-ID: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>

Why does this not work:

class User(ValidateUserPasswordMixin, PermissionsMixin, Entity,
AbstractBaseUser):
GENDER_UNKNOWN = 0
GENDER_FEMALE = 1
GENDER_MALE = 2
GENDER_OTHER = 3
GENDER_MAX_VALUE_PLUS_ONE = 4

GENDER_FEMALE_STRING = 'female'
GENDER_MALE_STRING = 'male'
GENDER_OTHER_STRING = 'other'

GENDER_CHOICES = (
(GENDER_FEMALE, _("Female")),
(GENDER_MALE, _("Male")),
(GENDER_OTHER, _("Other")),
)
GENDER_VALID_VALUES = [choice[0] for choice in GENDER_CHOICES]
GENDERS_DICT = {GENDER_FEMALE: GENDER_FEMALE_STRING, GENDER_MALE:
GENDER_MALE_STRING, GENDER_OTHER: GENDER_OTHER_STRING}
ALL_GENDERS = [GENDERS_DICT[gender] for gender in GENDER_VALID_VALUES]

(it throws an exception: `NameError: name 'GENDERS_DICT' is not defined`)

But this works:

class User(ValidateUserPasswordMixin, PermissionsMixin, Entity,
AbstractBaseUser):
GENDER_UNKNOWN = 0
GENDER_FEMALE = 1
GENDER_MALE = 2
GENDER_OTHER = 3
GENDER_MAX_VALUE_PLUS_ONE = 4

GENDER_FEMALE_STRING = 'female'
GENDER_MALE_STRING = 'male'
GENDER_OTHER_STRING = 'other'

GENDER_CHOICES = (
(GENDER_FEMALE, _("Female")),
(GENDER_MALE, _("Male")),
(GENDER_OTHER, _("Other")),
)
GENDER_VALID_VALUES = [choice[0] for choice in GENDER_CHOICES]
GENDERS_DICT = {GENDER_FEMALE: GENDER_FEMALE_STRING, GENDER_MALE:
GENDER_MALE_STRING, GENDER_OTHER: GENDER_OTHER_STRING}

User.ALL_GENDERS = [User.GENDERS_DICT[gender] for gender in
User.GENDER_VALID_VALUES]

Thanks,
???? (Uri)
uri at speedy.net


From fetchinson at googlemail.com  Tue Dec 25 07:53:53 2018
From: fetchinson at googlemail.com (Fetchinson .)
Date: Tue, 25 Dec 2018 13:53:53 +0100
Subject: building 3.7.1 from source, _ctypes and libffi troubles
In-Reply-To: <CADjSo4Tn19RE16+mycA8+D2igKNC1py4kq0MZtE0h7jNP1ZGFA@mail.gmail.com>
References: <CADjSo4SfUXb5oa9sHbFbV5GKAWdrJeMyCftVKO4xTdLmTpd+uw@mail.gmail.com>
 <vg3ftuqpmtu.fsf@coffee.modeemi.fi>
 <CADjSo4Tn19RE16+mycA8+D2igKNC1py4kq0MZtE0h7jNP1ZGFA@mail.gmail.com>
Message-ID: <CADjSo4S4+T+=VUBM1fmcCKvJqms_oZAdENgP0cJRrivpXJM24w@mail.gmail.com>

>> And as far as I know pkg-config is used by python's configure script
>> so everything should be fine. I also set
>> LD_LIBRARY_PATH=/home/fetch/opt/lib:/home/fetch/opt/lib64 and also
>> C_INCLUDE_PATH=/home/fetch/opt/include
>
> I looked into this a little. I found that setting C_INCLUDE_PATH as you
> did disables finding libffi headers since pkg-config prints nothing with
> pkg-config --cflags libffi then and it seems the headers are found with
> pkg-config.

This didn't actually occur for me, I had C_INCLUDE_PATH set and your
second suggestion (see below) worked.

> As for the libraries, detect_modules() in setup.py wants to find extra
> library directories from the generated makefile so you need to make sure
> they actually go in there, like this for example:
>
> LDFLAGS=`pkg-config --libs-only-L libffi` ./configure
>
> Then you'll have CONFIGURE_LDFLAGS and PY_LDFLAGS set in the generated
> makefile and setup.py will pick up the directory from there.

Thanks a lot! This indeed worked, now _ctypes compiles no problemlo!

Cheers,
Daniel


> Anyways, hope it helps. This worked for me on Ubuntu 16.04.5 LTS.


From tjreedy at udel.edu  Tue Dec 25 10:36:59 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 25 Dec 2018 10:36:59 -0500
Subject: Why doesn't a dictionary work in classes?
In-Reply-To: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>
References: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>
Message-ID: <pvtimp$4qu$1@blaine.gmane.org>

On 12/25/2018 7:45 AM, ???? wrote:
> Why does this not work:
> 
> class User(ValidateUserPasswordMixin, PermissionsMixin, Entity,
> AbstractBaseUser):
> GENDER_UNKNOWN = 0
 > ...

Indent needed here.  I presume you used tabs, which disappeared in 
transmission, but I cannot cut and paste to run this.



-- 
Terry Jan Reedy




From giuseppecostanzi at gmail.com  Tue Dec 25 11:41:11 2018
From: giuseppecostanzi at gmail.com (Beppe)
Date: Tue, 25 Dec 2018 08:41:11 -0800 (PST)
Subject: [ANN] Biovarase ver 4.2
Message-ID: <bbc792fb-5372-4f8c-9313-9f6125c4f051@googlegroups.com>

Biovarase has been updated to version 4.2

Biovarase is an application to manage clinical quality control data.

The purpose of Quality Control Assurance in a clinical laboratory is to allow the control of the performances of an analytical procedure showing an alarm as soon as the trial doesn't result in the degree to respect the defined analytical rules. Biovarase furthermore calculates the classical statistical parameters for the quality control assurance ,e.g. sd, cv%, avg, and even the Imp(%), Bias(%) and TEa (total allowable error) using data retrived from: Current databases on biologic variation: pros, cons and progress Scand J Clin Lab Invest 1999;59:491-500. updated with the most recent specifications made available in 2014.
It uses even the famous Westgard's rules to monitor results dataset.
All the data are managed by SQLite database and matplotlib.

Biovarase requires

    Python =>3.5
    tkinter
    matplotlib
    xlwt


last changelog

2018-12-25

Hi all and merry christmas, this is the new 4.2 version of Biovarase.

I' ve use this powerfull tool, Spyder, to refactoring all the project.

It's very cool;)

Change widgets.py to tools.py because it seems widgets it' s a reserved word in deep python tk file....

Clean, with the Spider help, very much code lines.

All source code on

https://github.com/1966bc/Biovarase

thank you for reading.

regards
beppe


From avigross at verizon.net  Tue Dec 25 11:59:49 2018
From: avigross at verizon.net (Avi Gross)
Date: Tue, 25 Dec 2018 11:59:49 -0500
Subject: Why doesn't a dictionary work in classes?
In-Reply-To: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>
References: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>
Message-ID: <000e01d49c73$3fdd0b10$bf972130$@verizon.net>

????,

Your indentation did not make it through. So the first thing to guess is what you are asking python to do.

It is often good to not just supply the code but point out what you are trying to do as well as explain the small difference you made in your code that made it work so we can zoom in on that.

It looks like you want to create a class but then you do nothing that follows up. If we indent further lines, it looks like you are creating constant values. The class is not initialized, there are no methods, etc.

Your class definition though brings in parent classes that presumably supply what you need so you are merely extending it by supplying a form of sorts, right?

	" class User(ValidateUserPasswordMixin, PermissionsMixin, Entity, AbstractBaseUser):"

I have no idea where those parent classes are defined and since they are the same in both the failing case and the working case, that should not matter. But if I have no access to that, I cannot easily replicate your code. I can try running some lines at the global level and I note some errors there:

What are the naked underscores doing in this:

GENDER_CHOICES = (
    (GENDER_FEMALE, _("Female")),
    (GENDER_MALE, _("Male")),
    (GENDER_OTHER, _("Other")),
    )

You seem to be making an odd tuple of tuples. Is the first tuple supposed to join the numerical representation of 1, with the character representation of "Female" to make (1,"Female") and the other two tuples something similar? You already declared similar (but all lowercase) strings above. So what is this odd notation for:
	_("Female")

If that section fails, then GENDER_CHOICES does not get created. Did you not get an error message for that?

But moving on to  your error message about not finding the dictionary in what shows up here as one merged line:

" GENDER_VALID_VALUES = [choice[0] for choice in GENDER_CHOICES] GENDERS_DICT = {GENDER_FEMALE: GENDER_FEMALE_STRING, GENDER_MALE:
GENDER_MALE_STRING, GENDER_OTHER: GENDER_OTHER_STRING} ALL_GENDERS = [GENDERS_DICT[gender] for gender in GENDER_VALID_VALUES]"

I see it as being better formatted as something like this:

    GENDER_VALID_VALUES = [choice[0] for choice in GENDER_CHOICES]
    GENDERS_DICT = {GENDER_FEMALE: GENDER_FEMALE_STRING,
                    GENDER_MALE: GENDER_MALE_STRING,
                    GENDER_OTHER: GENDER_OTHER_STRING}
    ALL_GENDERS = [GENDERS_DICT[gender] for gender in GENDER_VALID_VALUES]

The first line wants to make a list and it looks like you want a list of the first character in Male, Female and Other.

I assume you want to create a dict on the second line(s) looking like this: {1: 'female', 2: 'male', 3: 'other'}

I will not debate with you the choice of 3=2+1 as there are multiple variations on what you call other out there.

The last line, is trying to make a list of values from the dictionary just created. The error message you show suggests the dictionary creation failed on the lines above it.

So what is different in the version that worked?

    GENDER_VALID_VALUES = [choice[0] for choice in GENDER_CHOICES] 
    GENDERS_DICT = {GENDER_FEMALE: GENDER_FEMALE_STRING, 
		GENDER_MALE:	 GENDER_MALE_STRING, 
		GENDER_OTHER: GENDER_OTHER_STRING}

    User.ALL_GENDERS = [User.GENDERS_DICT[gender] for gender in User.GENDER_VALID_VALUES]

What you might have pointed out is that the last line was changed. It gives an expanded name to the variables you used. It is not clear due to the indentation problem if you ran this inside the class definition or outside.

Your variable names are all upper case and do not seem likely to cause confusion with the other parent classes but what about AbstractBaseUser? Is it possible that you may be trying to access a name defined in both contexts and need to carefully spell out which one you want here?

Sorry for the length of my message. I needed to work my way through piece by piece. As always, I might have done it differently so half the task is trying to think more like the writer.

???


-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On Behalf Of ????
Sent: Tuesday, December 25, 2018 7:46 AM
To: python-list at python.org
Subject: Why doesn't a dictionary work in classes?

Why does this not work:

class User(ValidateUserPasswordMixin, PermissionsMixin, Entity,
AbstractBaseUser):
GENDER_UNKNOWN = 0
GENDER_FEMALE = 1
GENDER_MALE = 2
GENDER_OTHER = 3
GENDER_MAX_VALUE_PLUS_ONE = 4

GENDER_FEMALE_STRING = 'female'
GENDER_MALE_STRING = 'male'
GENDER_OTHER_STRING = 'other'

GENDER_CHOICES = (
(GENDER_FEMALE, _("Female")),
(GENDER_MALE, _("Male")),
(GENDER_OTHER, _("Other")),
)
GENDER_VALID_VALUES = [choice[0] for choice in GENDER_CHOICES] GENDERS_DICT = {GENDER_FEMALE: GENDER_FEMALE_STRING, GENDER_MALE:
GENDER_MALE_STRING, GENDER_OTHER: GENDER_OTHER_STRING} ALL_GENDERS = [GENDERS_DICT[gender] for gender in GENDER_VALID_VALUES]

(it throws an exception: `NameError: name 'GENDERS_DICT' is not defined`)

But this works:

class User(ValidateUserPasswordMixin, PermissionsMixin, Entity,
AbstractBaseUser):
GENDER_UNKNOWN = 0
GENDER_FEMALE = 1
GENDER_MALE = 2
GENDER_OTHER = 3
GENDER_MAX_VALUE_PLUS_ONE = 4

GENDER_FEMALE_STRING = 'female'
GENDER_MALE_STRING = 'male'
GENDER_OTHER_STRING = 'other'

GENDER_CHOICES = (
(GENDER_FEMALE, _("Female")),
(GENDER_MALE, _("Male")),
(GENDER_OTHER, _("Other")),
)
GENDER_VALID_VALUES = [choice[0] for choice in GENDER_CHOICES] GENDERS_DICT = {GENDER_FEMALE: GENDER_FEMALE_STRING, GENDER_MALE:
GENDER_MALE_STRING, GENDER_OTHER: GENDER_OTHER_STRING}

User.ALL_GENDERS = [User.GENDERS_DICT[gender] for gender in User.GENDER_VALID_VALUES]

Thanks,
???? (Uri)
uri at speedy.net
--
https://mail.python.org/mailman/listinfo/python-list



From eryksun at gmail.com  Tue Dec 25 16:03:04 2018
From: eryksun at gmail.com (eryk sun)
Date: Tue, 25 Dec 2018 15:03:04 -0600
Subject: Why doesn't a dictionary work in classes?
In-Reply-To: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>
References: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>
Message-ID: <CACL+1asUhruZ_O_MZRmiDMa8kKG2rH=FAU+_PALQ8e9PQ-ciHw@mail.gmail.com>

On 12/25/18, ???? <uri at speedy.net> wrote:
>
> ALL_GENDERS = [GENDERS_DICT[gender] for gender in GENDER_VALID_VALUES]
>
> (it throws an exception: `NameError: name 'GENDERS_DICT' is not defined`)

Python 3 comprehensions have their own scope. This prevents the side
effect of defining the comprehension's loop variable in the enclosing
scope, which is a problem in Python 2. However, it also prevents the
evaluated expression from directly accessing names defined in an
unoptimized enclosing scope, except for the global scope, since such
scopes cannot be the source scope of closure variables. This applies
to comprehensions in class statements and exec(), where globals and
locals are different dicts.

Our 'parameter' for this comprehension scope is the expression for the
outer loop iterable, which gets evaluated in the enclosing scope.
Thus, for what it's worth, we can hack this to work as follows:

    ALL_GENDERS = [GD[gender]
                    for GD, GVV in ((GENDERS_DICT, GENDER_VALID_VALUES),)
                        for gender in GVV]


From __peter__ at web.de  Tue Dec 25 16:09:30 2018
From: __peter__ at web.de (Peter Otten)
Date: Tue, 25 Dec 2018 22:09:30 +0100
Subject: Why doesn't a dictionary work in classes?
References: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>
Message-ID: <pvu667$ljf$1@blaine.gmane.org>

???? wrote:

> Why does this not work:
> 
> class User(ValidateUserPasswordMixin, PermissionsMixin, Entity,
> AbstractBaseUser):

>     GENDER_VALID_VALUES = [choice[0] for choice in GENDER_CHOICES]
>     GENDERS_DICT = {GENDER_FEMALE: GENDER_FEMALE_STRING, GENDER_MALE:

>     ALL_GENDERS = [GENDERS_DICT[gender] for gender in GENDER_VALID_VALUES]
> 
> (it throws an exception: `NameError: name 'GENDERS_DICT' is not defined`)

The list comprehension in Python 3 works like a generator generator 
expression:

https://www.python.org/dev/peps/pep-0289/?modTest=#the-details

In your case GENDER_VALID_VALUES is looked up in the class, and GENDERS_DICT 
is looked up in the function synthesized from the list comprehension.
As with methods this function only sees the local and the global namespace, 
but not the class namespace.

I realise that this is a long-winded way of saying "it is like it is",
but maybe it helps you understand what works and what doesn't.
This would work for example (not recommended for actual use):

    ALL_GENDERS = [
        GENDERS_DICT[gender] for gender, GENDERS_DICT in
        zip(GENDER_VALID_VALUES, itertools.repeat(GENDERS_DICT))
    ]


> But this works:

> User.ALL_GENDERS = [User.GENDERS_DICT[gender] for gender in
> User.GENDER_VALID_VALUES]

User exists in the global namespace, and has both attributes, so no 
surprises here.



From uri at speedy.net  Wed Dec 26 00:10:32 2018
From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=)
Date: Wed, 26 Dec 2018 07:10:32 +0200
Subject: Why doesn't a dictionary work in classes?
In-Reply-To: <pvu667$ljf$1@blaine.gmane.org>
References: <CABD5YeFtc9=OWegkRr8ENNBSVTX2jmjDZg9+syto1LLzdHrxGA@mail.gmail.com>
 <pvu667$ljf$1@blaine.gmane.org>
Message-ID: <CABD5YeF3bXS-__DFYKrWYN4ZEOnC07Cu7Q=t-kVVoTVqH0WeWg@mail.gmail.com>

Hi,

Thank you all for your replies. After the first reply (about indentation) I
submitted a similar question on Stack Overflow, and I found out there that
a similar question has been asked before and I saw the answer there. The
indentation looks better on Stack Overflow than in email messages,
sorry about that.
https://stackoverflow.com/questions/53924235/why-doesnt-a-dictionary-work-in-classes

By the way, I remember having a problem in the past when writing a list
comprehension which uses a variable name that was also used outside
the list comprehension. for example "[i * 2 for i in range(n)]" when i is
used outside. This had the side effect of changing the variable i and it
took me a long time to debug it since I didn't expect it. But I think then
it was Python 2.7 and I'm glad it's been fixed in Python 3. But, I didn't
expect a list comprehension not to be able to access variables in this
scope. I also think this is a bug in Python. And by the way, I used list
comprehensions above too. For example, [choice[0] for choice in
GENDER_CHOICES] is also a list comprehension. It is not intuitive why
some list comprehensions work and others don't work.

If you want to see the full code with the branch I'm currently working on,
you can check
https://github.com/speedy-net/speedy-net/tree/uri_change_settings_2018-12-25_a
- for example this code is on
https://github.com/speedy-net/speedy-net/blob/uri_change_settings_2018-12-25_a/speedy/core/accounts/models.py

??? (Avi), there are methods but I didn't include them in this email
because I thought they are not relevant to my question.

I used upper case for constants as I understand this is the convention.
Actually, I did consider to use all lowercase Python (including class names
and constants) but another programmer told me it's not the way Python is
usually written.

I also tried to use "ALL_GENDERS = [__class__.GENDERS_DICT[gender] for
gender in __class__.GENDER_VALID_VALUES]" but this also didn't work there.

I don't like defining User.ALL_GENDERS after the end of the definition
of class User and as far as I remember this is the only time I defined a
class like that, but I didn't think I have any choice here. I don't want to
hack the definition as you suggested. I know since this list contains only
3 elements, I could define them specifically. But I didn't want to do it.
You can see in the tests that I defined it in 3 ways and tested they are
identical:

        self.assertListEqual(list1=User.ALL_GENDERS,
list2=[User.GENDERS_DICT[gender] for gender in User.GENDER_VALID_VALUES])
        self.assertListEqual(list1=User.ALL_GENDERS,
list2=[User.GENDER_FEMALE_STRING, User.GENDER_MALE_STRING,
User.GENDER_OTHER_STRING])
        self.assertListEqual(list1=User.ALL_GENDERS, list2=['female',
'male', 'other'])


Thanks,
???? (Uri)
uri at speedy.net


From frank at chagford.com  Wed Dec 26 01:17:52 2018
From: frank at chagford.com (Frank Millman)
Date: Wed, 26 Dec 2018 08:17:52 +0200
Subject: Complex 'compare'
Message-ID: <pvv6af$fk9$1@blaine.gmane.org>

"Frank Millman"  wrote on 2018-12-18 in message news:...
>
> I want to compare two tuples. They each represent a row in a database, and 
> each element represents a column, so I will use that terminology.
>
> I need to know if one row is greater than or less than the other. The sort 
> sequence can be complex - one or more columns, each of which can be sorted 
> ascending or descending.
>
> Below is the function I have come up with. Can anyone see any problem with 
> it, or suggest a better way to do it?
>

I know this is of minor interest, but I have refined my function and it 
looks much neater now, so I thought I would share it.

def compare(source_row, target_row, order):

    # source_row - the row I want to compare
    #                    - some sort columns could contain None
    # target_row - the row I want to compare it with
    #                    - no sort columns will contain None
    # order - a list of 2-part tuples defining the sort sequence
    #          - the column number, and True if descending else False

    LT, EQ, GT = -1, 0, 1

    for pos, desc in order:
        source_col = source_row[pos]
        target_col = target_row[pos]
        if desc:
            less_than = False if source_col is None else source_col > 
target_col
        else:
            less_than = True if source_col is None else source_col < 
target_col
        if less_than:
            return LT
        if new_data != col_data:
            return GT
        # must be EQ - continue with next sort column
    return EQ  # if we get here, all sort columns compare equal




From scopensource at gmail.com  Wed Dec 26 10:35:12 2018
From: scopensource at gmail.com (Simon Connah)
Date: Wed, 26 Dec 2018 15:35:12 +0000
Subject: Rate limiting a web crawler
Message-ID: <f89a9591-2c29-c875-c67a-f54e9c687f74@gmail.com>

Hi,

I want to build a simple web crawler. I know how I am going to do it but 
I have one problem.

Obviously I don't want to negatively impact any of the websites that I 
am crawling so I want to implement some form of rate limiting of HTTP 
requests to specific domain names.

What I'd like is some form of timer which calls a piece of code say 
every 5 seconds or something and that code is what goes off and crawls 
the website.

I'm just not sure on the best way to call code based on a timer.

Could anyone offer some advice on the best way to do this? It will be 
running on Linux and using the python-daemon library to run it as a 
service and will be using at least Python 3.6.

Thanks for any help.


From Richard at Damon-Family.org  Wed Dec 26 13:30:40 2018
From: Richard at Damon-Family.org (Richard Damon)
Date: Wed, 26 Dec 2018 13:30:40 -0500
Subject: Rate limiting a web crawler
In-Reply-To: <f89a9591-2c29-c875-c67a-f54e9c687f74@gmail.com>
References: <f89a9591-2c29-c875-c67a-f54e9c687f74@gmail.com>
Message-ID: <899992a4-f723-e241-2403-7e06c9804195@Damon-Family.org>

On 12/26/18 10:35 AM, Simon Connah wrote:
> Hi,
>
> I want to build a simple web crawler. I know how I am going to do it
> but I have one problem.
>
> Obviously I don't want to negatively impact any of the websites that I
> am crawling so I want to implement some form of rate limiting of HTTP
> requests to specific domain names.
>
> What I'd like is some form of timer which calls a piece of code say
> every 5 seconds or something and that code is what goes off and crawls
> the website.
>
> I'm just not sure on the best way to call code based on a timer.
>
> Could anyone offer some advice on the best way to do this? It will be
> running on Linux and using the python-daemon library to run it as a
> service and will be using at least Python 3.6.
>
> Thanks for any help.

One big piece of information that would help in replies would be an
indication of scale. Is you application crawling just a few sites, so
that you need to pause between accesses to keep the hit rate down, or
are you calling a number of sites, so that if you are going to delay
crawling a page from one site, you can go off and crawl another in the
mean time?

-- 
Richard Damon



From tjreedy at udel.edu  Wed Dec 26 14:04:14 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 26 Dec 2018 14:04:14 -0500
Subject: Rate limiting a web crawler
In-Reply-To: <f89a9591-2c29-c875-c67a-f54e9c687f74@gmail.com>
References: <f89a9591-2c29-c875-c67a-f54e9c687f74@gmail.com>
Message-ID: <q00j7c$f5v$1@blaine.gmane.org>

On 12/26/2018 10:35 AM, Simon Connah wrote:
> Hi,
> 
> I want to build a simple web crawler. I know how I am going to do it but 
> I have one problem.
> 
> Obviously I don't want to negatively impact any of the websites that I 
> am crawling so I want to implement some form of rate limiting of HTTP 
> requests to specific domain names.
> 
> What I'd like is some form of timer which calls a piece of code say 
> every 5 seconds or something and that code is what goes off and crawls 
> the website.
> 
> I'm just not sure on the best way to call code based on a timer.
> 
> Could anyone offer some advice on the best way to do this? It will be 
> running on Linux and using the python-daemon library to run it as a 
> service and will be using at least Python 3.6.

You can use asyncio to make repeated non-blocking requests to a web site 
at timed intervals and to work with multiple websites at once.  You can 
do the same with tkinter except that requests would block until a 
response unless you implemented your own polling.

-- 
Terry Jan Reedy



From scopensource at gmail.com  Wed Dec 26 14:32:11 2018
From: scopensource at gmail.com (Simon Connah)
Date: Wed, 26 Dec 2018 19:32:11 +0000
Subject: Rate limiting a web crawler
In-Reply-To: <899992a4-f723-e241-2403-7e06c9804195@Damon-Family.org>
References: <f89a9591-2c29-c875-c67a-f54e9c687f74@gmail.com>
 <899992a4-f723-e241-2403-7e06c9804195@Damon-Family.org>
Message-ID: <d189830a-37d3-c98e-7814-898e70034a9b@gmail.com>

On 26/12/2018 18:30, Richard Damon wrote:
> On 12/26/18 10:35 AM, Simon Connah wrote:
>> Hi,
>>
>> I want to build a simple web crawler. I know how I am going to do it
>> but I have one problem.
>>
>> Obviously I don't want to negatively impact any of the websites that I
>> am crawling so I want to implement some form of rate limiting of HTTP
>> requests to specific domain names.
>>
>> What I'd like is some form of timer which calls a piece of code say
>> every 5 seconds or something and that code is what goes off and crawls
>> the website.
>>
>> I'm just not sure on the best way to call code based on a timer.
>>
>> Could anyone offer some advice on the best way to do this? It will be
>> running on Linux and using the python-daemon library to run it as a
>> service and will be using at least Python 3.6.
>>
>> Thanks for any help.
> 
> One big piece of information that would help in replies would be an
> indication of scale. Is you application crawling just a few sites, so
> that you need to pause between accesses to keep the hit rate down, or
> are you calling a number of sites, so that if you are going to delay
> crawling a page from one site, you can go off and crawl another in the
> mean time?
> 

Sorry. I should have stated that.

This is for a minimum viable product so crawling say two or three domain 
names would be enough to start with but I'd want to grow in the future.

I'm building this on AWS and my idea was to have each web crawler 
instance query a database (DynamoDB) and get say 10 URLs and if they 
hadn't be crawled in the previous say 12 to 24 hours then recrawl them. 
If they have been crawled in the last 12 to 24 hours then skip that URL. 
Once a URL has been crawled I would then save the crawl date and time in 
the database.

Doing it that way I could skip the whole timing thing on the daemon end 
and just use database queries to control whether a URL is crawled or 
not. Of course that would mean that one web crawler would have to "lock" 
a domain name so that multiple instances do not query the same domain 
name in parallel which would be bad.


From scopensource at gmail.com  Wed Dec 26 14:34:07 2018
From: scopensource at gmail.com (Simon Connah)
Date: Wed, 26 Dec 2018 19:34:07 +0000
Subject: Rate limiting a web crawler
In-Reply-To: <q00j7c$f5v$1@blaine.gmane.org>
References: <f89a9591-2c29-c875-c67a-f54e9c687f74@gmail.com>
 <q00j7c$f5v$1@blaine.gmane.org>
Message-ID: <0d4542f4-ad83-042d-feb0-5b6693116c5d@gmail.com>

On 26/12/2018 19:04, Terry Reedy wrote:
> On 12/26/2018 10:35 AM, Simon Connah wrote:
>> Hi,
>>
>> I want to build a simple web crawler. I know how I am going to do it 
>> but I have one problem.
>>
>> Obviously I don't want to negatively impact any of the websites that I 
>> am crawling so I want to implement some form of rate limiting of HTTP 
>> requests to specific domain names.
>>
>> What I'd like is some form of timer which calls a piece of code say 
>> every 5 seconds or something and that code is what goes off and crawls 
>> the website.
>>
>> I'm just not sure on the best way to call code based on a timer.
>>
>> Could anyone offer some advice on the best way to do this? It will be 
>> running on Linux and using the python-daemon library to run it as a 
>> service and will be using at least Python 3.6.
> 
> You can use asyncio to make repeated non-blocking requests to a web site 
> at timed intervals and to work with multiple websites at once.? You can 
> do the same with tkinter except that requests would block until a 
> response unless you implemented your own polling.
> 

Thank you. I'll look into asynio.


From gunasekarr2000 at gmail.com  Wed Dec 26 14:37:29 2018
From: gunasekarr2000 at gmail.com (Gunasekar Rajendran)
Date: Wed, 26 Dec 2018 11:37:29 -0800 (PST)
Subject: Pycharm issue with import ssl
Message-ID: <e0c963f7-6c4b-4dac-8e78-ead4d6c5718e@googlegroups.com>

While trying to run the Python code in PyCharm 2018.3.2 version I am getting the below error. Can someone help?

Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.2\helpers\pydev\pydevconsole.py", line 5, in <module>
    from _pydev_comm.rpc import make_rpc_client, start_rpc_server, start_rpc_server_and_make_client
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.2\helpers\pydev\_pydev_comm\rpc.py", line 4, in <module>
    from _pydev_comm.server import TSingleThreadedServer
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.2\helpers\pydev\_pydev_comm\server.py", line 4, in <module>
    from _shaded_thriftpy.server import TServer
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.2\helpers\third_party\thriftpy\_shaded_thriftpy\server.py", line 9, in <module>
    from _shaded_thriftpy.transport import (
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.2\helpers\third_party\thriftpy\_shaded_thriftpy\transport\__init__.py", line 57, in <module>
    from .sslsocket import TSSLSocket, TSSLServerSocket  # noqa
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.2\helpers\third_party\thriftpy\_shaded_thriftpy\transport\sslsocket.py", line 7, in <module>
    import ssl
  File "C:\Users\grajendran\Anaconda3\lib\ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: DLL load failed: The specified module could not be found.
Process finished with exit code 1


From jfong at ms4.hinet.net  Wed Dec 26 21:53:25 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Wed, 26 Dec 2018 18:53:25 -0800 (PST)
Subject: Ask for help about class variable scope (Re: Why doesn't a dictionary
 work in classes?)
Message-ID: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>

I saw the code below at stackoverflow. I have a little idea about the scope of a class, and list comprehension and generator expressions, but still can't figure out why Z4 works and Z5 not. Can someone explain it? (in a not-too-complicated way:-)

class Foo():
    XS = [15, 15, 15, 15]
    Z4 = sum(val for val in XS)
    try:
        Z5 = sum(XS[i] for i in range(len(XS)))
    except NameError:
        Z5 = None

print(Foo.Z4, Foo.Z5)
>>> 60 None


--Jach


From arj.python at gmail.com  Thu Dec 27 01:07:06 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Thu, 27 Dec 2018 10:07:06 +0400
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
Message-ID: <CADrxXXnnex35c5M-U9ErcvjFP8NOi+LdzsoAyfV7=Z9r7hCCaw@mail.gmail.com>

greetings,

1)

Z4 = sum(val for val in XS)

is same as

Z4 = sum(XS)

2)

class Foo()

can also ne written as

class Foo:

3)

in Foo.x you are using the class just to assoxiate some variables with a
name. what is the purpose of tge script / what are you trying to do?

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius


From rosuav at gmail.com  Thu Dec 27 01:18:30 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 27 Dec 2018 17:18:30 +1100
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
Message-ID: <CAPTjJmpqfJG+YmmaS=S6cJHL76jazoiBF0JqD_J-Q2eo3uT-QQ@mail.gmail.com>

On Thu, Dec 27, 2018 at 1:56 PM <jfong at ms4.hinet.net> wrote:
>
> I saw the code below at stackoverflow. I have a little idea about the scope of a class, and list comprehension and generator expressions, but still can't figure out why Z4 works and Z5 not. Can someone explain it? (in a not-too-complicated way:-)
>
> class Foo():
>     XS = [15, 15, 15, 15]
>     Z4 = sum(val for val in XS)
>     try:
>         Z5 = sum(XS[i] for i in range(len(XS)))
>     except NameError:
>         Z5 = None
>
> print(Foo.Z4, Foo.Z5)
> >>> 60 None
>

Class scope is special, and a generator expression within that class
scope is special too. There have been proposals to make these kinds of
things less special, but the most important thing to remember is that
when you create a generator expression, it is actually a function.
Remember that a function inside a class statement becomes a method,
and that inside the method, you have to use "self.X" rather than just
"X" to reference class attributes. That's what's happening here.

ChrisA


From eryksun at gmail.com  Thu Dec 27 01:31:43 2018
From: eryksun at gmail.com (eryk sun)
Date: Thu, 27 Dec 2018 00:31:43 -0600
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
Message-ID: <CACL+1avwhLj5Q9ygM_VEV7veL-7-Yc7Ewb8nzYnknLVYMUW2Kw@mail.gmail.com>

On 12/26/18, jfong at ms4.hinet.net <jfong at ms4.hinet.net> wrote:
> I saw the code below at stackoverflow. I have a little idea about the scope
> of a class, and list comprehension and generator expressions, but still
> can't figure out why Z4 works and Z5 not. Can someone explain it? (in a
> not-too-complicated way:-)
>
> class Foo():
>     XS = [15, 15, 15, 15]
>     Z4 = sum(val for val in XS)
>     try:
>         Z5 = sum(XS[i] for i in range(len(XS)))
>     except NameError:
>         Z5 = None
>
> print(Foo.Z4, Foo.Z5)
>>>> 60 None

Maybe rewriting it with approximately equivalent inline code and
generator functions will clarify the difference:

    class Foo:
        def genexpr1(iterable):
            for val in iterable:
                yield val

        def genexpr2(iterable):
            for i in iterable:
                yield XS[i]

        XS = [15, 15, 15, 15]
        Z4 = sum(genexpr1(XS))
        try:
            Z5 = sum(genexpr2(range(len(XS))))
        except NameError:
            Z5 = None

        del genexpr1, genexpr2

    >>> print(Foo.Z4, Foo.Z5)
    60 None

In both cases, an iterable is passed to the generator function. This
argument is evaluated in the calling scope (e.g. range(len(XS))). A
generator expression has a similar implementation, except it also
evaluates the iterator for the iterable to ensure an exception is
raised immediately in the defining scope if it's not iterable. For
example:

    >>> (x for x in 1)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: 'int' object is not iterable

genexpr1 is working with local variables only, but genexpr2 has a
non-local reference to variable XS, which we call late binding. In
this case, when the generator code executes the first pass of the loop
(whenever that is), it looks for XS in the global (module) scope and
builtins scope. It's not there, so a NameError is raised.

With late-binding, the variable can get deleted or modified in the
source scope while the generator gets evaluated. For example:

    >>> x = 'spam'
    >>> g = (x[i] for i in range(len(x)))
    >>> next(g)
    's'
    >>> del x
    >>> next(g)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 1, in <genexpr>
    NameError: name 'x' is not defined

    >>> x = 'spam'
    >>> g = (x[i] for i in range(len(x)))
    >>> next(g)
    's'
    >>> x = 'eggs'
    >>> list(g)
    ['g', 'g', 's']


From jfong at ms4.hinet.net  Thu Dec 27 04:23:06 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Thu, 27 Dec 2018 01:23:06 -0800 (PST)
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <mailman.25.1545892306.2779.python-list@python.org>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
 <CACL+1avwhLj5Q9ygM_VEV7veL-7-Yc7Ewb8nzYnknLVYMUW2Kw@mail.gmail.com>
 <mailman.25.1545892306.2779.python-list@python.org>
Message-ID: <b5f17aec-ed07-4607-959e-bbe981e6dab0@googlegroups.com>

eryk sun? 2018?12?27???? UTC+8??2?31?58????
> On 12/26/18, jfong at ms4.hinet.net <jfong at ms4.hinet.net> wrote:
> > I saw the code below at stackoverflow. I have a little idea about the scope
> > of a class, and list comprehension and generator expressions, but still
> > can't figure out why Z4 works and Z5 not. Can someone explain it? (in a
> > not-too-complicated way:-)
> >
> > class Foo():
> >     XS = [15, 15, 15, 15]
> >     Z4 = sum(val for val in XS)
> >     try:
> >         Z5 = sum(XS[i] for i in range(len(XS)))
> >     except NameError:
> >         Z5 = None
> >
> > print(Foo.Z4, Foo.Z5)
> >>>> 60 None
> 
> Maybe rewriting it with approximately equivalent inline code and
> generator functions will clarify the difference:
> 
>     class Foo:
>         def genexpr1(iterable):
>             for val in iterable:
>                 yield val
> 
>         def genexpr2(iterable):
>             for i in iterable:
>                 yield XS[i]
> 
>         XS = [15, 15, 15, 15]
>         Z4 = sum(genexpr1(XS))
>         try:
>             Z5 = sum(genexpr2(range(len(XS))))
>         except NameError:
>             Z5 = None
> 
>         del genexpr1, genexpr2
> 
>     >>> print(Foo.Z4, Foo.Z5)
>     60 None
> 
> In both cases, an iterable is passed to the generator function. This
> argument is evaluated in the calling scope (e.g. range(len(XS))). A
> generator expression has a similar implementation, except it also
> evaluates the iterator for the iterable to ensure an exception is
> raised immediately in the defining scope if it's not iterable. For
> example:
> 
>     >>> (x for x in 1)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: 'int' object is not iterable
> 
> genexpr1 is working with local variables only, but genexpr2 has a
> non-local reference to variable XS, which we call late binding. In
> this case, when the generator code executes the first pass of the loop
> (whenever that is), it looks for XS in the global (module) scope and
> builtins scope. It's not there, so a NameError is raised.
> 
> With late-binding, the variable can get deleted or modified in the
> source scope while the generator gets evaluated. For example:
> 
>     >>> x = 'spam'
>     >>> g = (x[i] for i in range(len(x)))
>     >>> next(g)
>     's'
>     >>> del x
>     >>> next(g)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>       File "<stdin>", line 1, in <genexpr>
>     NameError: name 'x' is not defined
> 
>     >>> x = 'spam'
>     >>> g = (x[i] for i in range(len(x)))
>     >>> next(g)
>     's'
>     >>> x = 'eggs'
>     >>> list(g)
>     ['g', 'g', 's']

I still don't get it. When I change it to using list comprehension, the problem is still there. (it now has no late-binding variable, right? :-)

>>> class Too:
...     XS = [15, 15, 15, 15]
...     Z4 = [val for val in XS]
...     Z5 = [XS[0] for val in XS]
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in Too
  File "<stdin>", line 4, in <listcomp>
NameError: name 'XS' is not defined
>>>

The problem confuse me is that is XS a local variable of the list comprehension?
If it's then Z5 should work, if it's not then Z4 shouldn't work when it's written in generator expression.

Or, things is much more complex than I thought:-(




From tjreedy at udel.edu  Thu Dec 27 05:43:59 2018
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 27 Dec 2018 05:43:59 -0500
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
Message-ID: <q02a9d$le2$1@blaine.gmane.org>

On 12/26/2018 9:53 PM, jfong at ms4.hinet.net wrote:
> I saw the code below at stackoverflow. I have a little idea about the scope of a class, and list comprehension and generator expressions, but still can't figure out why Z4 works and Z5 not. Can someone explain it? (in a not-too-complicated way:-)
> 
> class Foo():
>      XS = [15, 15, 15, 15]
>      Z4 = sum(val for val in XS)

The iterable passed in to the comprehension is XS.

>      try:
>          Z5 = sum(XS[i] for i in range(len(XS)))

The iterable passed in to the comprehension function is range(len(XS)). 
XS is a class name, not a global or outer function local name, hence 
invisible to the inner function.

>      except NameError:
>          Z5 = None
> 
> print(Foo.Z4, Foo.Z5)
>>>> 60 None
> 
> 
> --Jach
> 


-- 
Terry Jan Reedy



From eryksun at gmail.com  Thu Dec 27 05:58:17 2018
From: eryksun at gmail.com (eryk sun)
Date: Thu, 27 Dec 2018 04:58:17 -0600
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <b5f17aec-ed07-4607-959e-bbe981e6dab0@googlegroups.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
 <CACL+1avwhLj5Q9ygM_VEV7veL-7-Yc7Ewb8nzYnknLVYMUW2Kw@mail.gmail.com>
 <mailman.25.1545892306.2779.python-list@python.org>
 <b5f17aec-ed07-4607-959e-bbe981e6dab0@googlegroups.com>
Message-ID: <CACL+1auHhjpUMdsWQ2UuYt8nT2mre8TruEyWZEY4EGBUmDc8fA@mail.gmail.com>

On 12/27/18, jfong at ms4.hinet.net <jfong at ms4.hinet.net> wrote:
>
> I still don't get it. When I change it to using list comprehension, the
> problem is still there. (it now has no late-binding variable, right? :-)
>
>>>> class Too:
> ...     XS = [15, 15, 15, 15]
> ...     Z4 = [val for val in XS]
> ...     Z5 = [XS[0] for val in XS]
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 4, in Too
>   File "<stdin>", line 4, in <listcomp>
> NameError: name 'XS' is not defined
>
> The problem confuse me is that is XS a local variable of the list
> comprehension?

XS is not a local variable in the scope of either comprehension. XS is
local to the class statement's scope. For each comprehension, an
iterator for the current object referenced by XS gets instantiated
(early binding) and passed as an argument to the comprehension scope.
If we disassemble the comprehension code, we find that this iterator
argument has the creatively illegal name ".0". (The bytecode
references it by its fast-locals-array index, not its weird name.)

In the Z5 case, XS is a non-local variable (late binding) in the
loop-body expression (left-hand side) of the comprehension. That's
common for Python code. In every iteration of the loop, the
interpreter looks up the object referenced by the name "XS", which can
change at any time (e.g. by another thread).


From ant at anthive.com  Thu Dec 27 08:15:37 2018
From: ant at anthive.com (ant)
Date: Thu, 27 Dec 2018 08:15:37 -0500
Subject: next steps to make a python program more available
Message-ID: <psveff-bk5.ln1@anthive.com>

  ok, i have the program going and installable for my
local Linux distribution.  aka i have it packaged and
on PyPI and it runs from my local terminal command line.

  i have people available for testing it out on Windows
and a Mac, but i don't have any way of knowing what to
do to make the program appear in their Menu system to
make it clickable.

  in MATE (which is what i run here) i can figure it out
as something to do with the desktop, so somehow i have
to do that, but i don't know what keywords to even search
for to do that, desktop icons i guess and menu, but is
there a more global way to set that up?  XDG seems to be
a part of that.

  i'm figuring if i get it working for MATE that it should
also work for Gnome.  i haven't touched KDE in quite a 
few years.

  Windows i'll figure out after the more Linux/Posix 
systems.

  so my priorties are about like:

  1. posix/other linux sytems (perhaps Macs fit into this anways)
  2. Macs
  3. Windows

  luckily i do have other examples of python 3 programs which
seem to be multiple platform oriented that i can look at and
see what they've done.  so i won't be forever lost in the
wilderness...

  thanks for help,  :)


  ant


From hjp-python at hjp.at  Thu Dec 27 08:17:34 2018
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Thu, 27 Dec 2018 14:17:34 +0100
Subject: Why do data descriptors (e.g. properties) take priority over
 instance attributes?
In-Reply-To: <CAAR6NGkwR0ZZFyc2=2=Xox7e+NEutSOKfQ78AitzM3AAz+Jnqw@mail.gmail.com>
References: <CAAR6NGkwR0ZZFyc2=2=Xox7e+NEutSOKfQ78AitzM3AAz+Jnqw@mail.gmail.com>
Message-ID: <20181227131734.huk7ckvqt74hhpzm@hjp.at>

On 2018-12-17 21:57:22 +1100, Paul Baker wrote:
>     class C:
>         def __init__(self):
>             self.__dict__['a'] = 1
> 
>         @property
>         def a(self):
>             return 2
> 
>     o = C()
>     print(o.a)  # Prints 2

What version of Python is this? I get a different result:

    Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
    [GCC 8.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> class C:
    ...   def __init__(self):
    ...     self.__dict__['a'] = 1
    ...     @property
    ...     def a(self):
    ...       return 2
    ... 
    >>> o = C()
    >>> o.a
    1

	hp

-- 
   _  | Peter J. Holzer    | we build much bigger, better disasters now
|_|_) |                    | because we have much more sophisticated
| |   | hjp at hjp.at         | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20181227/5b3878c8/attachment.sig>

From hjp-python at hjp.at  Thu Dec 27 08:24:12 2018
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Thu, 27 Dec 2018 14:24:12 +0100
Subject: Why do data descriptors (e.g. properties) take priority over
 instance attributes?
In-Reply-To: <20181227131734.huk7ckvqt74hhpzm@hjp.at>
References: <CAAR6NGkwR0ZZFyc2=2=Xox7e+NEutSOKfQ78AitzM3AAz+Jnqw@mail.gmail.com>
 <20181227131734.huk7ckvqt74hhpzm@hjp.at>
Message-ID: <20181227132412.qg46ym32wpzqhy6n@hjp.at>

On 2018-12-27 14:17:34 +0100, Peter J. Holzer wrote:
> On 2018-12-17 21:57:22 +1100, Paul Baker wrote:
> >     class C:
> >         def __init__(self):
> >             self.__dict__['a'] = 1
> > 
> >         @property
> >         def a(self):
> >             return 2
> > 
> >     o = C()
> >     print(o.a)  # Prints 2
> 
> What version of Python is this? I get a different result:
> 
>     Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
>     [GCC 8.2.0] on linux
>     Type "help", "copyright", "credits" or "license" for more information.
>     >>> class C:
>     ...   def __init__(self):
>     ...     self.__dict__['a'] = 1
>     ...     @property
>     ...     def a(self):
>     ...       return 2
>     ... 
>     >>> o = C()
>     >>> o.a
>     1

Oops. I messed up the indentation. 

With the same indentation I get the same result.

        hp

-- 
   _  | Peter J. Holzer    | we build much bigger, better disasters now
|_|_) |                    | because we have much more sophisticated
| |   | hjp at hjp.at         | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20181227/bf9fbd95/attachment.sig>

From sandeep.bayi6 at gmail.com  Thu Dec 27 09:19:03 2018
From: sandeep.bayi6 at gmail.com (sandeep.bayi6 at gmail.com)
Date: Thu, 27 Dec 2018 06:19:03 -0800 (PST)
Subject: Getting Error after migrating from python 3.4.1 to python 3.6.6 (
 Failed to import the site module )
Message-ID: <583142f7-25ca-44dc-8aaf-2f21116a45ba@googlegroups.com>

```````````````````````````````````````````````````````````````````````````
Error code:
--------------------------------------------------------------------------


Traceback (most recent call last):
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\site.py", line 73, in <module
    import os
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\os.py", line 652, in <module>
    from _collections_abc import MutableMapping
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\_collections_abc.py", line 58
    async def _coro(): pass
            ^
SyntaxError: invalid syntax
Failed to import the site module

````````````````````````````````````````````````````````````````````````````
After migrating from python 3.4.1 to python 3.6.1
while Executing my project, I'm facing this issue and not able to resolve it



From sandeep.bayi6 at gmail.com  Thu Dec 27 09:23:28 2018
From: sandeep.bayi6 at gmail.com (sandeep.bayi6 at gmail.com)
Date: Thu, 27 Dec 2018 06:23:28 -0800 (PST)
Subject: Getting Error after migrating from python 3.4.1 to python 3.6.6 (
 Failed to import the site module )
In-Reply-To: <583142f7-25ca-44dc-8aaf-2f21116a45ba@googlegroups.com>
References: <583142f7-25ca-44dc-8aaf-2f21116a45ba@googlegroups.com>
Message-ID: <e646e1ee-e447-4aef-a0e4-71e17460f07c@googlegroups.com>

On Thursday, December 27, 2018 at 7:49:16 PM UTC+5:30, sandee... at gmail.com wrote:
> ```````````````````````````````````````````````````````````````````````````
> Error code:
> --------------------------------------------------------------------------
> 
> 
> Traceback (most recent call last):
>   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\site.py", line 73, in <module
>     import os
>   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\os.py", line 652, in <module>
>     from _collections_abc import MutableMapping
>   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\_collections_abc.py", line 58
>     async def _coro(): pass
>             ^
> SyntaxError: invalid syntax
> Failed to import the site module
> 
> ````````````````````````````````````````````````````````````````````````````
> After migrating from python 3.4.1 to python 3.6.6
> while Executing my project, I'm facing this issue and not able to resolve it. Can i find any solution for this error?



From sandeep.bayi6 at gmail.com  Thu Dec 27 09:26:57 2018
From: sandeep.bayi6 at gmail.com (sandeep.bayi6 at gmail.com)
Date: Thu, 27 Dec 2018 06:26:57 -0800 (PST)
Subject: Facing an Error after migrating from python 3.4.1 to python 3.6.6 (
 Failed to import the site module  )
Message-ID: <79ca23c8-2a95-40eb-b3a8-0887a5803765@googlegroups.com>



``````````````````````````````````````````````````````````````````````````` 
Error code: 
-------------------------------------------------------------------------- 


Traceback (most recent call last): 
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\site.py", line 73, in <module 
    import os 
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\os.py", line 652, in <module> 
    from _collections_abc import MutableMapping 
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\_collections_abc.py", line 58 
    async def _coro(): pass 
            ^ 
SyntaxError: invalid syntax 
Failed to import the site module 

```````````````````````````````````````````````````````````````````````````` 
After migrating from python 3.4.1 to python 3.6.6 
while Executing my project, I'm facing this issue and not able to resolve it. Can i get any solution for this issue? 


From ian.g.kelly at gmail.com  Thu Dec 27 11:29:38 2018
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Thu, 27 Dec 2018 09:29:38 -0700
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <CAPTjJmpqfJG+YmmaS=S6cJHL76jazoiBF0JqD_J-Q2eo3uT-QQ@mail.gmail.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
 <CAPTjJmpqfJG+YmmaS=S6cJHL76jazoiBF0JqD_J-Q2eo3uT-QQ@mail.gmail.com>
Message-ID: <CALwzidnh_Ni456fFoPZ5w6qn4Lhx_1xLPmcdbU1e-84s9AWmqg@mail.gmail.com>

On Wed, Dec 26, 2018 at 11:21 PM Chris Angelico <rosuav at gmail.com> wrote:
>
> On Thu, Dec 27, 2018 at 1:56 PM <jfong at ms4.hinet.net> wrote:
> >
> > I saw the code below at stackoverflow. I have a little idea about the scope of a class, and list comprehension and generator expressions, but still can't figure out why Z4 works and Z5 not. Can someone explain it? (in a not-too-complicated way:-)
> >
> > class Foo():
> >     XS = [15, 15, 15, 15]
> >     Z4 = sum(val for val in XS)
> >     try:
> >         Z5 = sum(XS[i] for i in range(len(XS)))
> >     except NameError:
> >         Z5 = None
> >
> > print(Foo.Z4, Foo.Z5)
> > >>> 60 None
> >
>
> Class scope is special, and a generator expression within that class
> scope is special too. There have been proposals to make these kinds of
> things less special, but the most important thing to remember is that
> when you create a generator expression, it is actually a function.
> Remember that a function inside a class statement becomes a method,
> and that inside the method, you have to use "self.X" rather than just
> "X" to reference class attributes. That's what's happening here.

Except you can't use "self" either because the class doesn't exist yet
at the time the generator expression is being evaluated. Nor can you
use "self" generally even if you wait until the class does exist
before you iterate over the generator, because while a generator
expression may scope like a function, and may under the hood be
implemented as a function, a generator object is not a function object
and will not create a method.


From rosuav at gmail.com  Thu Dec 27 12:37:55 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Dec 2018 04:37:55 +1100
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <CALwzidnh_Ni456fFoPZ5w6qn4Lhx_1xLPmcdbU1e-84s9AWmqg@mail.gmail.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
 <CAPTjJmpqfJG+YmmaS=S6cJHL76jazoiBF0JqD_J-Q2eo3uT-QQ@mail.gmail.com>
 <CALwzidnh_Ni456fFoPZ5w6qn4Lhx_1xLPmcdbU1e-84s9AWmqg@mail.gmail.com>
Message-ID: <CAPTjJmpYgKs+oESSO5wogmqrD7MKgR7mkY3exPCPyB73WH369g@mail.gmail.com>

On Fri, Dec 28, 2018 at 3:31 AM Ian Kelly <ian.g.kelly at gmail.com> wrote:
>
> On Wed, Dec 26, 2018 at 11:21 PM Chris Angelico <rosuav at gmail.com> wrote:
> >
> > On Thu, Dec 27, 2018 at 1:56 PM <jfong at ms4.hinet.net> wrote:
> > >
> > > I saw the code below at stackoverflow. I have a little idea about the scope of a class, and list comprehension and generator expressions, but still can't figure out why Z4 works and Z5 not. Can someone explain it? (in a not-too-complicated way:-)
> > >
> > > class Foo():
> > >     XS = [15, 15, 15, 15]
> > >     Z4 = sum(val for val in XS)
> > >     try:
> > >         Z5 = sum(XS[i] for i in range(len(XS)))
> > >     except NameError:
> > >         Z5 = None
> > >
> > > print(Foo.Z4, Foo.Z5)
> > > >>> 60 None
> > >
> >
> > Class scope is special, and a generator expression within that class
> > scope is special too. There have been proposals to make these kinds of
> > things less special, but the most important thing to remember is that
> > when you create a generator expression, it is actually a function.
> > Remember that a function inside a class statement becomes a method,
> > and that inside the method, you have to use "self.X" rather than just
> > "X" to reference class attributes. That's what's happening here.
>
> Except you can't use "self" either because the class doesn't exist yet
> at the time the generator expression is being evaluated.

Well, yes. But that's the easiest way to highlight the difference of scope.

> Nor can you
> use "self" generally even if you wait until the class does exist
> before you iterate over the generator, because while a generator
> expression may scope like a function, and may under the hood be
> implemented as a function, a generator object is not a function object
> and will not create a method.

Hmm. It creates a function which is called at class scope. Based on
disassembly, I can't see a difference between the function created for
a genexp and the one created for a method. What IS true, though, is
that there's no way to pass arguments to the genexp, which means that
it won't be passed a 'self'. So if you wait till the class exists
before iterating over the generator, you can name the class, but you
can't use self:

>>> class Foo:
...     word = "spam"
...     genexp = (l.upper() for shim in [...] for l in Foo.word)
...
>>> list(Foo.genexp)
['S', 'P', 'A', 'M']

I would personally have preferred for the official Python language
definition to have described comprehensions and genexps as creating a
scope boundary, without baking into the language "they're hidden,
implicit functions". But that ship has sailed, and when I debated the
point, all Pandora's box got opened.

ChrisA


From avigross at verizon.net  Thu Dec 27 16:45:59 2018
From: avigross at verizon.net (Avi Gross)
Date: Thu, 27 Dec 2018 16:45:59 -0500
Subject: dangerous class neighborhood
Message-ID: <005f01d49e2d$8ed49380$ac7dba80$@verizon.net>

There have been several discussions about unexpected behavior when people
write programs within a class definition. Some suggest various ways around
the problem and that is fine although I bet new people will keep
encountering this issue.

 

I have some questions about what people want to do with this kind of
programming and for each use, there may be suggested ways to get things done
and get the right result in a reasonably natural way.

 

I have some overall concerns regarding the aggressive approach taken in
Python toward supporting too many features in what is loosely considered
object-oriented programming to the point where some combinations interact in
mathematically intractable ways resulting in concessions and compromises
such as the one we see.

 

Question 1: Do you want evaluation at class definition time or at run time?

 

Question 2: Do you want the variables available at the class level or at the
instance level?

 

Question 3: Which python variations on syntactic sugar, such as list
comprehensions, get expanded invisibly in ways that make the problem happen
by asking for variables to be found when no longer in the visible range?

 

There may be matters of efficiency some would consider but some of the
examples seen recently seemed almost silly and easy to compute. The people
asking about this issue wanted to define a bunch of CONSTANTS, or things
that might as well be constants, like this:

 

def Foo():

                A = ("male", "female", "other")

                B = [ kind[0] for kind in A ]            # First letters
only

                # And so on making more constants like a dictionary mapping
each string to a number or vice versa.

 

All the above can be evaluated at the time the class is defined but
unintuitive scope rules make some operations fail as variables defined in
the scope become unavailable to other things that SEEM to be embedded in the
same scope.

 

So, are these constants being called as Foo.A or are they used after an
instance is created like:

 

Aleph = Foo()

# Use Aleph.A 

 

If they are ONLY to be used within an instance of Foo or invoked from within
there, there may be a fairly simple suggestion. If you already have a
__init__ method, then instantiate the variables there carefully using the
self object to reference those needed.

 

def Foo():

                def __init__(self):

                                # calculate A and B and whatever.

                                self.A = A

self.B = B

# .

These calculations would not be at declaration time for class Foo but the
results would be created every time an object of class Foo came along. If
Foo is subclassed, it would be wise to make sure the initialization reaches
up to run Foo.__init__ properly too.

 

Within __init__ I presume all the USUAL scoping rules make sense so if
making a B includes using A, it would easily be found there first. Again,
not self.A, just A at that stage. You can create self.A near the end of the
initialization when all calculations needed are complete.

 

Second suggestion, even if what is being made may not be constants. 

 

Create a function either outside the class or defined within. Have it do any
internal calculations you need in which all internal variables can play
nicely with each other. Then let it return all the variables in a tuple like
this:

 

def make_sexual_constants():

                A = .

                B = .

                C = f(A,B)

                D = .

def Foo():

                (A, B, C, D) = make_sexual_constants():

 

Can we agree that the class Foo now has those 4 variables defined and
available at either the class level or sub-class or instance levels? But the
values are created, again, in a unified safe environment?

 

As noted in section 3, it would be good to know what python features may be
unsafe in this kind of context. I had an unrelated recent discussion where
it was mentioned that some proposed feature changes might not be thread
safe. Valid consideration when that may lead to hard-to-explain anomalies.  

 

We now hear that because a list comprehension can be unwound internally into
a "while" loop and an "if" statement and that some parts may expand to calls
to a "range" statement, perhaps some variables are now in more deeply
embedded contexts that have no access to any class variables. I think that
is quite reasonable; hence my suggestion we need to know which ones to
avoid, or use a workaround like expanding it out ourselves and perhaps
carefully import variables into other contexts such as by passing the
variable into the function that otherwise cannot access it from a point it
can still be seen.

 

Least, but at least last, I ask if the need really exists for these
variables as constants versus functions. If creating C this way runs into
problems, but A and B are fine, consider making a method with some name like
Foo.get_C() that can see A and B and do the calculation and yet return the
value of C needed. Less efficient but .

 

I apologize for the length but assure you I normally would say much more.

 

Avi

 

 

 

 



From eryksun at gmail.com  Thu Dec 27 16:46:15 2018
From: eryksun at gmail.com (eryk sun)
Date: Thu, 27 Dec 2018 15:46:15 -0600
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <CAPTjJmpqfJG+YmmaS=S6cJHL76jazoiBF0JqD_J-Q2eo3uT-QQ@mail.gmail.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
 <CAPTjJmpqfJG+YmmaS=S6cJHL76jazoiBF0JqD_J-Q2eo3uT-QQ@mail.gmail.com>
Message-ID: <CACL+1asgbRb9Zc3cCeqq0_suFcP6LX_Z7Z_1Oj2HXpw6nXr2rQ@mail.gmail.com>

On 12/27/18, Chris Angelico <rosuav at gmail.com> wrote:
>
> Class scope is special, and a generator expression within that class
> scope is special too. There have been proposals to make these kinds of
> things less special, but the most important thing to remember is that
> when you create a generator expression, it is actually a function.
> Remember that a function inside a class statement becomes a method,
> and that inside the method, you have to use "self.X" rather than just
> "X" to reference class attributes. That's what's happening here.

A generator expression is implemented internally as a generator
function that takes an iterator as its only parameter (named ".0") and
gets called immediately to get a generator object. There's some inline
bytecode in the defining scope that sets this up.

A generator object has iterator methods (__iter__, __next__) and
close, send, and throw methods. Its code and execution state in
CPython uses a code object and a frame object:

    >>> g = (c for c in 'spam')
    >>> g.gi_code.co_varnames
    ('.0', 'c')

Initially it hasn't run, so there's no 'c' value yet:

    >>> sorted(g.gi_frame.f_locals)
    ['.0']

'c' is defined after it executes up to the first yield:

    >>> next(g)
    's'
    >>> sorted(g.gi_frame.f_locals)
    ['.0', 'c']

Unlike a function object, a generator object is not a descriptor (i.e.
it has no __get__ method) that could in principle be bound as either a
class or instance method. Anyway, since the class doesn't exist yet,
trying to bind and call a method at this point can't work.

In contrast generator functions are commonly used for methods that can
access class attributes. But this is confusing matters since the class
__dict__ (and certainly not the instance __dict__) that's used for
attributes is not the locals() of the initial class statement
execution. In other words, class attribute access is not a closure
over the class statement scope. For methods, it depends on early
binding of the bound object to a method's __self__ attribute, which is
implicitly passed as the first argument to its __func__ function when
the method is called.

The execution locals of a class statement is a temporary namespace
that gets copied when the class object is instantiated. For example:

    class C:
        exec_dict = locals()

    >>> C.s = 'spam'
    >>> sorted(C.__dict__)
    ['__dict__', '__doc__', '__module__', '__weakref__', 'exec_dict', 's']

The original locals():

    >>> sorted(C.exec_dict)
    ['__module__', '__qualname__', 'exec_dict']

What could be done is for the compiler to introduce nonlocal free
variables (like what's already done with __class__), and then capture
the current value to the locals() dict after it's done executing. For
example (a clumsy one; in practice it would be implicit, expanding on
how __class__ is implemented):

    def make_Foo():
        XS = None
        class Foo:
            nonlocal XS
            XS = [15] * 4
            Z5 = sum(XS[i] for i in range(len(XS)))
            locals()['XS'] = XS
        return Foo

    >>> Foo = make_Foo()
    >>> Foo.Z5
    60
    >>> Foo.XS
    [15, 15, 15, 15]

However, this would be confusing in general, since there's no way to
keep the class attribute in sync with the cell variable. So a function
that's called as a class method may see different values for XS and
cls.XS. This is a bad idea.

This is straying off topic, but note that a consequence of late
binding is that super() can be broken by rebinding __class__:

    class C:
        def f(self):
            nonlocal __class__
            __class__ = None
            super()

    >>> C().f()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 5, in f
    RuntimeError: super(): __class__ is not a type (NoneType)

It's not a bug or design flaw; just an example of code shooting itself
in the foot by stepping on an informally reserved dunder name.


From rosuav at gmail.com  Thu Dec 27 17:10:42 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Dec 2018 09:10:42 +1100
Subject: dangerous class neighborhood
In-Reply-To: <005f01d49e2d$8ed49380$ac7dba80$@verizon.net>
References: <005f01d49e2d$8ed49380$ac7dba80$@verizon.net>
Message-ID: <CAPTjJmpyxNQeUuzMh3qo9f6VQj2dnvsWz0TTyNk38S75Ji4mWg@mail.gmail.com>

On Fri, Dec 28, 2018 at 8:47 AM Avi Gross <avigross at verizon.net> wrote:
> Question 2: Do you want the variables available at the class level or at the
> instance level?

For constants, definitely put them on the class. They'll be available
on instances as well ("for free", if you like). For mutables,
obviously you need to decide on a case-by-case basis.

> Question 3: Which python variations on syntactic sugar, such as list
> comprehensions, get expanded invisibly in ways that make the problem happen
> by asking for variables to be found when no longer in the visible range?

The oddities with comprehensions were tackled partly during the
discussion of PEP 572. If you want to know exactly why this isn't
changing, go read a few hundred emails on the subject. A lot of the
main points are summarized in the PEP itself:

https://www.python.org/dev/peps/pep-0572/

> There may be matters of efficiency some would consider but some of the
> examples seen recently seemed almost silly and easy to compute. The people
> asking about this issue wanted to define a bunch of CONSTANTS, or things
> that might as well be constants, like this:
>
>
>
> def Foo():
>
>                 A = ("male", "female", "other")
>
>                 B = [ kind[0] for kind in A ]            # First letters
> only
>
>                 # And so on making more constants like a dictionary mapping
> each string to a number or vice versa.
>
>
>
> All the above can be evaluated at the time the class is defined but
> unintuitive scope rules make some operations fail as variables defined in
> the scope become unavailable to other things that SEEM to be embedded in the
> same scope.

If you write simple and Pythonic code, these will almost always work
perfectly. The recent thread citing an oddity worked just fine until
it was written to iterate over range(len(x)) instead of iterating
directly.

> If they are ONLY to be used within an instance of Foo or invoked from within
> there, there may be a fairly simple suggestion. If you already have a
> __init__ method, then instantiate the variables there carefully using the
> self object to reference those needed.

But why? __init__ should initialize an instance, not class-level
constants. A Python class is not restricted to just methods, and
there's no reason to avoid class attributes.

> Create a function either outside the class or defined within. Have it do any
> internal calculations you need in which all internal variables can play
> nicely with each other. Then let it return all the variables in a tuple like
> this:
>
> def make_sexual_constants():
>
>                 A = .
>
>                 B = .
>
>                 C = f(A,B)
>
>                 D = .
>
> def Foo():
>
>                 (A, B, C, D) = make_sexual_constants():

Lovely. Now you have to define your variables once inside the
function, then name them a second time in that function's return
statement, and finally name them all a *third* time in the class
statement (at least, I presume "def Foo():" is meant to be "class
Foo:"). A mismatch will create bizarre and hard-to-debug problems.
What do you actually gain? Can you show me real-world code that would
truly benefit from this?

> Can we agree that the class Foo now has those 4 variables defined and
> available at either the class level or sub-class or instance levels? But the
> values are created, again, in a unified safe environment?

Unified? No more so than the class statement itself. Safe? Definitely
not, because of the mandatory duplication of names.

> As noted in section 3, it would be good to know what python features may be
> unsafe in this kind of context. I had an unrelated recent discussion where
> it was mentioned that some proposed feature changes might not be thread
> safe. Valid consideration when that may lead to hard-to-explain anomalies.

Uhh..... nope, that's nothing but FUD. There is no reason to believe
that some language features would be "unsafe".

> We now hear that because a list comprehension can be unwound internally into
> a "while" loop and an "if" statement and that some parts may expand to calls
> to a "range" statement, perhaps some variables are now in more deeply
> embedded contexts that have no access to any class variables.

No idea what you're looking at. A comprehension can be unwound in a
fairly straight-forward way, although there are some subtleties to
them.

B = [ kind[0] for kind in A ]
# equivalent to, approximately:
def listcomp(iter):
    result = []
    for kind in iter:
        result.append(kind[0])
    return result
B = listcomp(A)

For casual usage, you can describe a list comp very simply and neatly:

B = [ kind[0] for kind in A ]
# equivalent to, more approximately:
B = []
for kind in A:
    B.append(kind[0])

Nothing here expands to a call to range(), nothing has a while loop.
The only way you'll get an "if" is if you had one in the comprehension
itself.

> I think that
> is quite reasonable; hence my suggestion we need to know which ones to
> avoid, or use a workaround like expanding it out ourselves and perhaps
> carefully import variables into other contexts such as by passing the
> variable into the function that otherwise cannot access it from a point it
> can still be seen.

Sure. If the comprehension doesn't work for you, just put a for loop
inside your class statement. This is not a problem.

> Least, but at least last, I ask if the need really exists for these
> variables as constants versus functions. If creating C this way runs into
> problems, but A and B are fine, consider making a method with some name like
> Foo.get_C() that can see A and B and do the calculation and yet return the
> value of C needed. Less efficient but .

Definitely not. That would imply that the value of C might change, or
might have significant cost, or in some other way actually merits a
getter function. Python isn't built to encourage that.

Class scope has edge cases, to be sure, but they're much more notable
in carefully-crafted exploratory code than in actual real-world code.

ChrisA


From rosuav at gmail.com  Thu Dec 27 17:21:15 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Dec 2018 09:21:15 +1100
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <CACL+1asgbRb9Zc3cCeqq0_suFcP6LX_Z7Z_1Oj2HXpw6nXr2rQ@mail.gmail.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
 <CAPTjJmpqfJG+YmmaS=S6cJHL76jazoiBF0JqD_J-Q2eo3uT-QQ@mail.gmail.com>
 <CACL+1asgbRb9Zc3cCeqq0_suFcP6LX_Z7Z_1Oj2HXpw6nXr2rQ@mail.gmail.com>
Message-ID: <CAPTjJmpYfU2WMkWhwKXfmVb2+YaG+93RaGFmYKWLOr6PhfWaBQ@mail.gmail.com>

On Fri, Dec 28, 2018 at 8:47 AM eryk sun <eryksun at gmail.com> wrote:
>
> On 12/27/18, Chris Angelico <rosuav at gmail.com> wrote:
> >
> > Class scope is special, and a generator expression within that class
> > scope is special too. There have been proposals to make these kinds of
> > things less special, but the most important thing to remember is that
> > when you create a generator expression, it is actually a function.
> > Remember that a function inside a class statement becomes a method,
> > and that inside the method, you have to use "self.X" rather than just
> > "X" to reference class attributes. That's what's happening here.
>
> A generator expression is implemented internally as a generator
> function that takes an iterator as its only parameter (named ".0") and
> gets called immediately to get a generator object. There's some inline
> bytecode in the defining scope that sets this up.
>
> A generator object has iterator methods (__iter__, __next__) and
> close, send, and throw methods. Its code and execution state in
> CPython uses a code object and a frame object:

A generator expression is an expression that creates and then calls a
function to create a generator object. This is exactly the same as any
other generator function being called to create a generator object.
Yes, the generator function and the generator object are very
different beasts, and yes, they're both called "a generator", and yes,
this is confusing. But it's the same whether it's a genexp or an
explicit function.

> Unlike a function object, a generator object is not a descriptor (i.e.
> it has no __get__ method) that could in principle be bound as either a
> class or instance method. Anyway, since the class doesn't exist yet,
> trying to bind and call a method at this point can't work.

This is correct. A genexp created at class scope would be
approximately equivalent to a generator function called as "Foo.f()"
not as "Foo().f()" - called on the class, not an instance of it.

> This is straying off topic, but note that a consequence of late
> binding is that super() can be broken by rebinding __class__:
>
>     class C:
>         def f(self):
>             nonlocal __class__
>             __class__ = None
>             super()
>
>     >>> C().f()
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>       File "<stdin>", line 5, in f
>     RuntimeError: super(): __class__ is not a type (NoneType)
>
> It's not a bug or design flaw; just an example of code shooting itself
> in the foot by stepping on an informally reserved dunder name.

Technically only the no-arg form of super() can be broken that way,
because it's actually a bit of magic syntax that becomes
"super(__class__, self)", except that it doesn't actually use "self"
but "whatever the first argument to this function is". You can break
it in an even simpler way:

>>> class C:
...     def f():
...         super()
...
>>> C.f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in f
RuntimeError: super(): no arguments

"But Mum, I thought you said you wanted NO ARGUMENTS while you were out...."

ChrisA


From D.Ojalvo at F5.com  Thu Dec 27 18:47:05 2018
From: D.Ojalvo at F5.com (Daniel Ojalvo)
Date: Thu, 27 Dec 2018 23:47:05 +0000
Subject: Undocumented issue: Open system call blocks on named pipes (and a
 feature request)
Message-ID: <SN6PR01MB46874663F4911125B1B0E6B5BDB60@SN6PR01MB4687.prod.exchangelabs.com>

Hello,

I've been working on a python3 project and I came across an issue with the open system call that, at the very least, isn't documented. In my humble opinion, the documentation<https://docs.python.org/3/library/functions.html#open> should be updated because folks wouldn't expect open to be a blocking operation and simply error out. Worse yet, open doesn't have an option to make itself non-blocking. You have to use the os system calls to kludge a solution.

Here is how I reproduced the issue:

root at beefy:~/sandbox# mkfifo this_is_a_pipe
root at beefy:~/sandbox# ls -l this_is_a_pipe
prw-r--r-- 1 root root 0 Dec 27 14:28 this_is_a_pipe
root at beefy:~/sandbox# python3 --version
Python 3.6.7
root at beefy:~/sandbox# python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> open("this_is_a_pipe")
<blocks>

The mitigation is to use the os calls and specify to be nonblocking when opening the file.

I'm doing this to get a fileobject and make it error out if we do have a blocking special file:
with os.fdopen(os.open(<filename>, os.O_RDONLY| os.O_NONBLOCK) , mode='rb') as file_obj:

I think this is mostly a documentation bug because this wouldn't be expected behavior to someone reading the docs, but open is behaving as the fifo man page<http://man7.org/linux/man-pages/man7/fifo.7.html> is documented. The feature request would be to add a non-blocking option to the default open system call.

I've also found this with some named special character files, but I don't have a reproduction at the moment.

Thank you and have a good day!
Dan


From rosuav at gmail.com  Thu Dec 27 22:09:53 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Dec 2018 14:09:53 +1100
Subject: Undocumented issue: Open system call blocks on named pipes (and a
 feature request)
In-Reply-To: <SN6PR01MB46874663F4911125B1B0E6B5BDB60@SN6PR01MB4687.prod.exchangelabs.com>
References: <SN6PR01MB46874663F4911125B1B0E6B5BDB60@SN6PR01MB4687.prod.exchangelabs.com>
Message-ID: <CAPTjJmrFbEee5joZo2_h1_vOD=Ts1uzusJB4SMWJXYU1BJTYTQ@mail.gmail.com>

On Fri, Dec 28, 2018 at 1:38 PM Daniel Ojalvo via Python-list
<python-list at python.org> wrote:
>
> Hello,
>
> I've been working on a python3 project and I came across an issue with the open system call that, at the very least, isn't documented. In my humble opinion, the documentation<https://docs.python.org/3/library/functions.html#open> should be updated because folks wouldn't expect open to be a blocking operation and simply error out. Worse yet, open doesn't have an option to make itself non-blocking. You have to use the os system calls to kludge a solution.
>

Hmm. I disagree that the docs are deceptive here; I would normally
expect open() to block if it needs to. But looking at this as a
feature request, it seems reasonable. Actually, it's not even that
hard to do, since open() is already pluggable:

rosuav at sikorsky:~/tmp$ rm rene_magritte
rosuav at sikorsky:~/tmp$ mkfifo rene_magritte
rosuav at sikorsky:~/tmp$ ls -l rene_magritte
prw-r--r-- 1 rosuav rosuav 0 Dec 28 14:05 rene_magritte
rosuav at sikorsky:~/tmp$ python3
Python 3.8.0a0 (heads/master:8b9c33ea9c, Nov 20 2018, 02:18:50)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> def nonblock(fn, mode): return os.open(fn, mode | os.O_NONBLOCK)
...
>>> open("rene_magritte", opener=nonblock)
<_io.TextIOWrapper name='rene_magritte' mode='r' encoding='UTF-8'>
>>> _.read(1)
''

> Here is how I reproduced the issue:
>
> root at beefy:~/sandbox# mkfifo this_is_a_pipe

(my example file name is a more subtle reference...)

> I'm doing this to get a fileobject and make it error out if we do have a blocking special file:
> with os.fdopen(os.open(<filename>, os.O_RDONLY| os.O_NONBLOCK) , mode='rb') as file_obj:
>
> I think this is mostly a documentation bug because this wouldn't be expected behavior to someone reading the docs, but open is behaving as the fifo man page<http://man7.org/linux/man-pages/man7/fifo.7.html> is documented. The feature request would be to add a non-blocking option to the default open system call.
>

Honestly, I don't think there's a problem with it blocking by default.
Most of Python works that way. But it would be pretty straight-forward
to add "nonblocking=False" as another keyword-only parameter, and for
compatibility with existing versions (back as far as 3.3), the opener
should work just fine.

ChrisA


From avigross at verizon.net  Thu Dec 27 22:26:04 2018
From: avigross at verizon.net (Avi Gross)
Date: Thu, 27 Dec 2018 22:26:04 -0500
Subject: dangerous class neighborhood
Message-ID: <00cd01d49e5d$106b94a0$3142bde0$@verizon.net>

Sometimes when I post something I get back comments and evaluate them and
learn quite a bit. I then reply and debate every little point and it can
continue for a few rounds.

I don't seem to be in that mood today so let me simply restate my entire
post in a few sentences with no examples, no lectures, no advice on what
anyone else can do and very little for anyone to bother replying to. Here
goes:

Sometimes when I run up against a wall and find that a solution to a problem
does not work because things may not work as I expected, I pause. I
reconsider what I actually need to get done. Then I look to see if I can
come up with other ways to do it that will work while still getting the
important parts done. Failing that, I ask if perhaps there is another tool,
such as another programming language that is a better fit for the task. And,
if the work needed seems excessive, I ask if perhaps the problem does not
really need to be solved by me and I move on.

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Chris Angelico
Sent: Thursday, December 27, 2018 5:11 PM
To: Python <python-list at python.org>
Subject: Re: dangerous class neighborhood

On Fri, Dec 28, 2018 at 8:47 AM Avi Gross <avigross at verizon.net> wrote:
> Question 2: Do you want the variables available at the class level or 
> at the instance level?

For constants, definitely put them on the class. They'll be available on
instances as well ("for free", if you like). For mutables, obviously you
need to decide on a case-by-case basis.

> Question 3: Which python variations on syntactic sugar, such as list 
> comprehensions, get expanded invisibly in ways that make the problem 
> happen by asking for variables to be found when no longer in the 
> visible
range?

The oddities with comprehensions were tackled partly during the discussion
of PEP 572. If you want to know exactly why this isn't changing, go read a
few hundred emails on the subject. A lot of the main points are summarized
in the PEP itself:

https://www.python.org/dev/peps/pep-0572/

> There may be matters of efficiency some would consider but some of the 
> examples seen recently seemed almost silly and easy to compute. The 
> people asking about this issue wanted to define a bunch of CONSTANTS, 
> or things that might as well be constants, like this:
>
>
>
> def Foo():
>
>                 A = ("male", "female", "other")
>
>                 B = [ kind[0] for kind in A ]            # First letters
> only
>
>                 # And so on making more constants like a dictionary 
> mapping each string to a number or vice versa.
>
>
>
> All the above can be evaluated at the time the class is defined but 
> unintuitive scope rules make some operations fail as variables defined 
> in the scope become unavailable to other things that SEEM to be 
> embedded in the same scope.

If you write simple and Pythonic code, these will almost always work
perfectly. The recent thread citing an oddity worked just fine until it was
written to iterate over range(len(x)) instead of iterating directly.

> If they are ONLY to be used within an instance of Foo or invoked from 
> within there, there may be a fairly simple suggestion. If you already 
> have a __init__ method, then instantiate the variables there carefully 
> using the self object to reference those needed.

But why? __init__ should initialize an instance, not class-level constants.
A Python class is not restricted to just methods, and there's no reason to
avoid class attributes.

> Create a function either outside the class or defined within. Have it 
> do any internal calculations you need in which all internal variables 
> can play nicely with each other. Then let it return all the variables 
> in a tuple like
> this:
>
> def make_sexual_constants():
>
>                 A = .
>
>                 B = .
>
>                 C = f(A,B)
>
>                 D = .
>
> def Foo():
>
>                 (A, B, C, D) = make_sexual_constants():

Lovely. Now you have to define your variables once inside the function, then
name them a second time in that function's return statement, and finally
name them all a *third* time in the class statement (at least, I presume
"def Foo():" is meant to be "class Foo:"). A mismatch will create bizarre
and hard-to-debug problems.
What do you actually gain? Can you show me real-world code that would truly
benefit from this?

> Can we agree that the class Foo now has those 4 variables defined and 
> available at either the class level or sub-class or instance levels?
> But the values are created, again, in a unified safe environment?

Unified? No more so than the class statement itself. Safe? Definitely not,
because of the mandatory duplication of names.

> As noted in section 3, it would be good to know what python features 
> may be unsafe in this kind of context. I had an unrelated recent 
> discussion where it was mentioned that some proposed feature changes 
> might not be thread safe. Valid consideration when that may lead to
hard-to-explain anomalies.

Uhh..... nope, that's nothing but FUD. There is no reason to believe that
some language features would be "unsafe".

> We now hear that because a list comprehension can be unwound 
> internally into a "while" loop and an "if" statement and that some 
> parts may expand to calls to a "range" statement, perhaps some 
> variables are now in more deeply embedded contexts that have no access 
> to
any class variables.

No idea what you're looking at. A comprehension can be unwound in a fairly
straight-forward way, although there are some subtleties to them.

B = [ kind[0] for kind in A ]
# equivalent to, approximately:
def listcomp(iter):
    result = []
    for kind in iter:
        result.append(kind[0])
    return result
B = listcomp(A)

For casual usage, you can describe a list comp very simply and neatly:

B = [ kind[0] for kind in A ]
# equivalent to, more approximately:
B = []
for kind in A:
    B.append(kind[0])

Nothing here expands to a call to range(), nothing has a while loop.
The only way you'll get an "if" is if you had one in the comprehension
itself.

> I think that
> is quite reasonable; hence my suggestion we need to know which ones to 
> avoid, or use a workaround like expanding it out ourselves and perhaps 
> carefully import variables into other contexts such as by passing the 
> variable into the function that otherwise cannot access it from a 
> point it can still be seen.

Sure. If the comprehension doesn't work for you, just put a for loop inside
your class statement. This is not a problem.

> Least, but at least last, I ask if the need really exists for these 
> variables as constants versus functions. If creating C this way runs 
> into problems, but A and B are fine, consider making a method with 
> some name like
> Foo.get_C() that can see A and B and do the calculation and yet return 
> the value of C needed. Less efficient but .

Definitely not. That would imply that the value of C might change, or might
have significant cost, or in some other way actually merits a getter
function. Python isn't built to encourage that.

Class scope has edge cases, to be sure, but they're much more notable in
carefully-crafted exploratory code than in actual real-world code.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

_______________________________________________
Tutor maillist  -  Tutor at python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor



From rosuav at gmail.com  Fri Dec 28 00:08:15 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 28 Dec 2018 16:08:15 +1100
Subject: dangerous class neighborhood
In-Reply-To: <00cd01d49e5d$106b94a0$3142bde0$@verizon.net>
References: <00cd01d49e5d$106b94a0$3142bde0$@verizon.net>
Message-ID: <CAPTjJmpysLjQLqvR4+OUJe2nkkOxZ_38bO1vh2BT=2iPPBjQRg@mail.gmail.com>

On Fri, Dec 28, 2018 at 2:27 PM Avi Gross <avigross at verizon.net> wrote:
>
> Sometimes when I post something I get back comments and evaluate them and
> learn quite a bit. I then reply and debate every little point and it can
> continue for a few rounds.
>
> I don't seem to be in that mood today so let me simply restate my entire
> post in a few sentences with no examples, no lectures, no advice on what
> anyone else can do and very little for anyone to bother replying to. Here
> goes:
>
> Sometimes when I run up against a wall and find that a solution to a problem
> does not work because things may not work as I expected, I pause. I
> reconsider what I actually need to get done. Then I look to see if I can
> come up with other ways to do it that will work while still getting the
> important parts done. Failing that, I ask if perhaps there is another tool,
> such as another programming language that is a better fit for the task. And,
> if the work needed seems excessive, I ask if perhaps the problem does not
> really need to be solved by me and I move on.

So your entire original post can be summed up as "I don't understand
Python, and I don't care"? Why did you post it then? I was under the
impression that you wanted to learn what was going on.

ChrisA


From arj.python at gmail.com  Fri Dec 28 00:29:34 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 28 Dec 2018 09:29:34 +0400
Subject: dangerous class neighborhood
In-Reply-To: <005f01d49e2d$8ed49380$ac7dba80$@verizon.net>
References: <005f01d49e2d$8ed49380$ac7dba80$@verizon.net>
Message-ID: <CADrxXXknL5qZnMVyPr4Tx4SoSsR+Lf4_pc2bCXhS9Pt7ZaxQHw@mail.gmail.com>

btw what does

A = .

above means? the dot?

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius


From avigross at verizon.net  Fri Dec 28 01:35:45 2018
From: avigross at verizon.net (Avi Gross)
Date: Fri, 28 Dec 2018 01:35:45 -0500
Subject: dangerous class neighborhood
In-Reply-To: <CAPTjJmpysLjQLqvR4+OUJe2nkkOxZ_38bO1vh2BT=2iPPBjQRg@mail.gmail.com>
References: <00cd01d49e5d$106b94a0$3142bde0$@verizon.net>
 <CAPTjJmpysLjQLqvR4+OUJe2nkkOxZ_38bO1vh2BT=2iPPBjQRg@mail.gmail.com>
Message-ID: <010001d49e77$90411c30$b0c35490$@verizon.net>

Chris,

You pretty much did exactly what I expected. You took what I wrote and
replaced it with your words and suggested I had said that.

What I wrote this time did not mention python. By now I am far from being a
novice in python but do not claim deep experience or expertise either. In
time, perhaps. In many places I understand it well enough and after the many
replies and some earlier reading I do understand some aspects being
discussed. It is the fact that I do understand what I now think I do, and
ask questions to understand more, that lets me say that it is often a good
idea to bend when reality intrudes and find another way that IS supposed to
work. I posted a number of ways in an earlier message that seem like
reasonable alternatives to try IF your design needs are satisfied. Note that
what satisfies me might not satisfy you. Others have posted other ways we
can evaluate. There does not need to be just one way to do things. Sometimes
even a partial result that works almost all the time is also of value.

What I said next was conditional. FIRST find out why your initial method
fails. If it turns out it is because it is not legal use of python (or
whatever tool) then understand that and find methods that follow the rules.
If you cannot, and perhaps others cannot help you either, then maybe this is
not the right tool. Are you giving up at each stage? Sure. But giving up to
try something else is not the same as giving up and killing yourself. So I
said that when I encountered some problems, I reserve the flexibility to try
other paths to the point of switching tools. In the end, which can happen
upon repeated failures, I may decide it is not worth it.

Two examples and I stress these are examples and imply nothing that you may
choose to dump here.

One example is trying to solve a non-trivial problem using a computer when
any human on the street can solve it just by looking and your best algorithm
might work if run for a decade. Since you need to make a decision soon,
perhaps you can just look out the window and move on and solve other
problems that are more pressing and have no easy solution already available.
Perhaps you can revisit it some day when others have solved enough problems
with computer vision and other technologies and made changes to the legal
system so you can build a relatively safe car that drives itself while
killing few enough people. But doing that hundreds of years ago would not
have been easy, especially before there was a single car on the road. But
horses and carriages seemed available.

Another example is if you determine the problem is intractable. Can you make
an algorithm that simulates the entire universe (whatever that means) in
less than real time. I mean it keeps track of the initial position and
velocity of every particle (including neutrinos and photons and anything
else) to near-infinite precision in the entire  universe as well as the
curvature of space at every possible location in what may turn out to be an
infinite yet expanding realm and calculate how it will progress going
forward trillions of years while also calculating every possible outcome as
many events at the quantum level are in some deep sense probabilistic. Oh,
and do this on a computer that fits into a small room with a standard set of
computers, not some quantum computer. 

Most would agree that if you are working on your Ph.D. thesis and wish to
graduate, pick a different dissertation topic. The above probably can not be
solved with anything smaller than the entire universe which seemingly does
it in real time so asking for it to do it faster using less resources may be
a challenge. I have seen people fail with many smaller problems or find out
it had already been done by someone else and simply pick a new topic to work
on. That does not make them failures.

My point again is what I would do. I would stubbornly try various ways using
the first tool and on repeated failure see if perhaps there was a different
tool to use or build one or perhaps change the problem so it could be solved
and so on. But, ultimately, if daunted, I would not drop everything else in
my life but would indeed stop trying.

If you suggest you would do something better, feel free.

At some point a discussion or debate can switch to just insults and I
suggest I would prefer to have a meaningful interaction instead. I did learn
from this discussion and know ways I can write code to meet such needs. So I
fail to understand what you are saying. I make no assertions I can meet your
needs if they are not reasonable. No solution can exist if you want an
irresistible force to meet an unmovable object.

I suggest that if your attitude persists, you are the one making this a
dangerous neighborhood.

And further replies by me in public or private would serve no purpose. 

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Chris Angelico
Sent: Friday, December 28, 2018 12:08 AM
To: Python <python-list at python.org>
Subject: Re: dangerous class neighborhood

On Fri, Dec 28, 2018 at 2:27 PM Avi Gross <avigross at verizon.net> wrote:
>
> Sometimes when I post something I get back comments and evaluate them 
> and learn quite a bit. I then reply and debate every little point and 
> it can continue for a few rounds.
>
> I don't seem to be in that mood today so let me simply restate my 
> entire post in a few sentences with no examples, no lectures, no 
> advice on what anyone else can do and very little for anyone to bother 
> replying to. Here
> goes:
>
> Sometimes when I run up against a wall and find that a solution to a 
> problem does not work because things may not work as I expected, I 
> pause. I reconsider what I actually need to get done. Then I look to 
> see if I can come up with other ways to do it that will work while 
> still getting the important parts done. Failing that, I ask if perhaps 
> there is another tool, such as another programming language that is a 
> better fit for the task. And, if the work needed seems excessive, I 
> ask if perhaps the problem does not really need to be solved by me and I
move on.

So your entire original post can be summed up as "I don't understand Python,
and I don't care"? Why did you post it then? I was under the impression that
you wanted to learn what was going on.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list



From avigross at verizon.net  Fri Dec 28 02:01:07 2018
From: avigross at verizon.net (Avi Gross)
Date: Fri, 28 Dec 2018 02:01:07 -0500
Subject: dangerous class neighborhood
In-Reply-To: <CADrxXXknL5qZnMVyPr4Tx4SoSsR+Lf4_pc2bCXhS9Pt7ZaxQHw@mail.gmail.com>
References: <005f01d49e2d$8ed49380$ac7dba80$@verizon.net>
 <CADrxXXknL5qZnMVyPr4Tx4SoSsR+Lf4_pc2bCXhS9Pt7ZaxQHw@mail.gmail.com>
Message-ID: <010201d49e7b$1b747510$525d5f30$@verizon.net>

Short answer to a short question.

 

What does a dot (dot dot) mean in pseudocode?

 

NOTHING and EVERYTHING.

 

Longer answer:

 

I think what you see may not be what I wrote. I used three dots in a row (an ellipsis) to mean that something is left out to be filled in with something appropriate.

 

A = ?

B = ?

 

That was meant to mean that it does not really matter exactly what code is used. Set A equal to SOMETHING and set B equal to something.

 

As it happens, there are computer languages where that is valid code. Python turns out to be one of them!

 

 

>>> A=...

>>> A

Ellipsis

 

In python you can often use ?pass? in some contexts like:

 

>>> if 1: pass

>>> def name():

                Pass

 

These don?t do anything except serve as a placeholder and should be replaced later. There are cases where you can write something like this:

 

>>> x = 5

>>> if x > 7:

                pass

else:

                print(x)

 

                

5

 

But although that works, you could simply use a NOT or invert the condition to say:

 

>>> if not (x > 7):

                print(x)

 

                

5

>>> if (x <= 7):

                print(x)

 

                

5

 

Well ??? can also be used in modern versions of python although it may not be novice-level material. It can serve as a placeholder in things like this:

 

>>> def name(x,y,z): pass

 

>>> name(1,2,3)

>>> name(1,3)

Traceback (most recent call last):

  File "<pyshell#25>", line 1, in <module>

    name(1,3)

TypeError: name() missing 1 required positional argument: 'z'

>>> name(1,...,3)

 

You can see the third positional argument is still the third in this example:

 

>>> def name(x,y,z): print(x,z)

 

>>> name(1,...,3)

1 3

 

So what is the middle argument in this case?

 

>>> def name(x,y,z): print(x,y,z)

 

>>> name(1,...,3)

1 Ellipsis 3

>>> name(...,...,3)

Ellipsis Ellipsis 3

>>> name(...,...,...)

Ellipsis Ellipsis Ellipsis

 

I suspect that generally the intent is to eventually fill in the missing argument.

 

Here is a related anomaly. The single underscore has multiple uses but one is to say you are throwing away something.

 

>>> a, _, c = (1, "who cares", 3)

>>> a

1

>>> _

'who cares'

>>> c

3

 

But what I used the same variable name multiple times?

 

>>> _, _, c = (1, "who cares", 3)

>>> _

'who cares'

>>> c

3

 

To be fair, any variable used repeatedly in that context keeps only the rightmost assignment. But there is an idiom of using just an underscore to mean you don?t care about that and want something else whose name may be made meaningful.

 

>>> _, _, LastName = ("John", "J.", "Doe")

>>> LastName

'Doe'

 

So, my apologies. I won?t use ? even in pseudocode.

 

 

From: Abdur-Rahmaan Janhangeer <arj.python at gmail.com> 
Sent: Friday, December 28, 2018 12:30 AM
To: Avi Gross <avigross at verizon.net>
Cc: Python <python-list at python.org>
Subject: Re: dangerous class neighborhood

 

btw what does

 

A = .

 

above means? the dot?

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club <http://www.pythonmembersclub>  | https://github.com/Abdur-rahmaanJ
Mauritius



From jfong at ms4.hinet.net  Fri Dec 28 03:03:55 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Fri, 28 Dec 2018 00:03:55 -0800 (PST)
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <mailman.27.1545908300.2779.python-list@python.org>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
 <CACL+1avwhLj5Q9ygM_VEV7veL-7-Yc7Ewb8nzYnknLVYMUW2Kw@mail.gmail.com>
 <mailman.25.1545892306.2779.python-list@python.org>
 <b5f17aec-ed07-4607-959e-bbe981e6dab0@googlegroups.com>
 <CACL+1auHhjpUMdsWQ2UuYt8nT2mre8TruEyWZEY4EGBUmDc8fA@mail.gmail.com>
 <mailman.27.1545908300.2779.python-list@python.org>
Message-ID: <39b5b835-9960-4fca-a2ca-b07f72e3f6d2@googlegroups.com>

eryk sun at 2018/12/27 UTC+8 PM 6:58:33 wrote:
> On 12/27/18, jfong at ms4.hinet.net <jfong at ms4.hinet.net> wrote:
> >
> > I still don't get it. When I change it to using list comprehension, the
> > problem is still there. (it now has no late-binding variable, right? :-)
> >
> >>>> class Too:
> > ...     XS = [15, 15, 15, 15]
> > ...     Z4 = [val for val in XS]
> > ...     Z5 = [XS[0] for val in XS]
> > ...
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> >   File "<stdin>", line 4, in Too
> >   File "<stdin>", line 4, in <listcomp>
> > NameError: name 'XS' is not defined
> >
> > The problem confuse me is that is XS a local variable of the list
> > comprehension?
> 
> XS is not a local variable in the scope of either comprehension. XS is
> local to the class statement's scope. For each comprehension, an
> iterator for the current object referenced by XS gets instantiated
> (early binding) and passed as an argument to the comprehension scope.
> If we disassemble the comprehension code, we find that this iterator
> argument has the creatively illegal name ".0". (The bytecode
> references it by its fast-locals-array index, not its weird name.)
> 
> In the Z5 case, XS is a non-local variable (late binding) in the
> loop-body expression (left-hand side) of the comprehension. That's
> common for Python code. In every iteration of the loop, the
> interpreter looks up the object referenced by the name "XS", which can
> change at any time (e.g. by another thread).

In Python document 4.2.2. Resolution of names, the last paragraph:

    "...A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution with an exception that unbound local variables are looked up in the global namespace. ...The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods ? this includes comprehensions and generator expressions since they are implemented using a function scope". 

These statements reflect the following difference:

>>> xy = [1,2]
>>> [dir() for i in xy]
[['.0', 'i'], ['.0', 'i']]
>>> [xy[0] for i in xy]
[1, 1]

>>> class foo():
...     xs = [1,2]
...     z4 = [dir() for i in xs]
...
>>> foo().z4
[['.0', 'i'], ['.0', 'i']]
>>> class foo():
...     xs = [1,2]
...     z4 = [xs[0] for i in xs]
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in foo
  File "<stdin>", line 3, in <listcomp>
NameError: name 'xs' is not defined
>>>

and it goes further:

>>> [dir() for i in xy for j in xy]
[['.0', 'i', 'j'], ['.0', 'i', 'j'], ['.0', 'i', 'j'], ['.0', 'i', 'j']]
>>> class foo():
...     xs = [1,2]
...     z5 = [dir() for i in xs for j in xs]
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in foo
  File "<stdin>", line 3, in <listcomp>
NameError: name 'xs' is not defined
>>>

That's all I had learn so far, although not understand the design decision behind it yet:-(

--Jach



From arj.python at gmail.com  Fri Dec 28 03:45:54 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 28 Dec 2018 12:45:54 +0400
Subject: dangerous class neighborhood
In-Reply-To: <010201d49e7b$1b747510$525d5f30$@verizon.net>
References: <005f01d49e2d$8ed49380$ac7dba80$@verizon.net>
 <CADrxXXknL5qZnMVyPr4Tx4SoSsR+Lf4_pc2bCXhS9Pt7ZaxQHw@mail.gmail.com>
 <010201d49e7b$1b747510$525d5f30$@verizon.net>
Message-ID: <CADrxXXkzAkKewhLZsArED7mp_TQVBeWFz6J2Qc8kLi1dUaMJZQ@mail.gmail.com>

thanks, the 3 dots are more explicit, thought was another py trick

yours,

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius


From zhen.zhong at innolux.com  Fri Dec 28 05:03:09 2018
From: zhen.zhong at innolux.com (zhen.zhong at innolux.com)
Date: Fri, 28 Dec 2018 18:03:09 +0800
Subject: ANNOUNCE: TIB/Rendezvous module for python
Message-ID: <OF22D4BA8E.512E2EC0-ON48258371.0037279B-48258371.00373887@innolux.com>

Hi, There,

        I am an engineer in Taiwan and I was trying to install 
rvpython-2.1final.tar.gz  but it's built up in python 2.x which cannot be 
installed in python 3 or above.

        Is there any officially released version of rvpython  compatible 
with python 3 or above?

        Thank you, sincerely. 


==================================================
Zhen Zhong  ??
Innolux Corporation
TFT PIA CIM1-EA
TEL?037-586000#64734
?????????????????????160?
No.160,Kesyue Rd., Chunan Science Park,Miao-Li Country 350,Taiwan
E-mail?zhen.zhong at innolux.com
=================================================
================================================================
This e-mail and any files transmitted with it are CONFIDENTIAL and 
intended solely for the use of the intended recipient.  If you are not the 
intended recipient or the named addressee, please notify the sender 
immediately and delete this e-mail and any files transmitted with it from 
your system; you should not disseminate, distribute, or copy this e-mail 
and any files transmitted with it, or take any action in reliance on the 
contents of the said e-mail and files.
================================================================

From piet-l at vanoostrum.org  Fri Dec 28 07:11:40 2018
From: piet-l at vanoostrum.org (Piet van Oostrum)
Date: Fri, 28 Dec 2018 13:11:40 +0100
Subject: Facing an Error after migrating from python 3.4.1 to python 3.6.6
 ( Failed to import the site module )
References: <79ca23c8-2a95-40eb-b3a8-0887a5803765@googlegroups.com>
Message-ID: <lxbm55c0lf.fsf@cochabamba.vanoostrum.org>

sandeep.bayi6 at gmail.com writes:

> ``````````````````````````````````````````````````````````````````````````` 
> Error code: 
> -------------------------------------------------------------------------- 
>
>
> Traceback (most recent call last): 
>   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\site.py", line 73, in <module 
>     import os 
>   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\os.py", line 652, in <module> 
>     from _collections_abc import MutableMapping 
>   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\_collections_abc.py", line 58 
>     async def _coro(): pass 
>             ^ 
> SyntaxError: invalid syntax 
> Failed to import the site module 
>
> ```````````````````````````````````````````````````````````````````````````` 
> After migrating from python 3.4.1 to python 3.6.6 
> while Executing my project, I'm facing this issue and not able to
> resolve it. Can i get any solution for this issue?

Could it be that your PYTHONPATH environment variable is set to a directory in Python 3.4.1?

-- 
Piet van Oostrum <piet-l at vanoostrum.org>
WWW: http://piet.vanoostrum.org/
PGP key: [8DAE142BE17999C4]


From arj.python at gmail.com  Fri Dec 28 07:15:23 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 28 Dec 2018 16:15:23 +0400
Subject: graded randomness
Message-ID: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>

greetings,

let us say that i have a box of 5 balls,

green balls - 2 with probability 2/5
red balls 2 - with probability 2/5
blue balls 1 - with probability 1/5

how to program the selection so that the random choices reflect the
probabilities?

-- 
Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Garanti
sans virus. www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>


From ben.usenet at bsb.me.uk  Fri Dec 28 08:17:23 2018
From: ben.usenet at bsb.me.uk (Ben Bacarisse)
Date: Fri, 28 Dec 2018 13:17:23 +0000
Subject: graded randomness
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
 <mailman.44.1545999024.2779.python-list@python.org>
Message-ID: <875zvdx02k.fsf@bsb.me.uk>

Abdur-Rahmaan Janhangeer <arj.python at gmail.com> writes:

> let us say that i have a box of 5 balls,
>
> green balls - 2 with probability 2/5
> red balls 2 - with probability 2/5
> blue balls 1 - with probability 1/5
>
> how to program the selection so that the random choices reflect the
> probabilities?

>>> import random
>>> random.choice(["green", "green", "red", "red", "blue"])

Not a method that scales particularly well, but there's almost no
context to evaluate solutions.

-- 
Ben.


From python.list at tim.thechases.com  Fri Dec 28 08:23:39 2018
From: python.list at tim.thechases.com (Tim Chase)
Date: Fri, 28 Dec 2018 07:23:39 -0600
Subject: graded randomness
In-Reply-To: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
Message-ID: <20181228072339.3f6dac35@bigbox.christie.dr>

On 2018-12-28 16:15, Abdur-Rahmaan Janhangeer wrote:
> greetings,
> 
> let us say that i have a box of 5 balls,
> 
> green balls - 2 with probability 2/5
> red balls 2 - with probability 2/5
> blue balls 1 - with probability 1/5
> 
> how to program the selection so that the random choices reflect the
> probabilities?

You're looking for what are called "weighted choices" which the
random.choice() function provides as of Py3.6

https://docs.python.org/3/library/random.html#random.choices

>>> from random import choices
>>> distribution = {"green":2, "red": 2, "blue", 1}
>>> data, weights = zip(*distribution.items())
>>> sum(weights)
5
>>> sorted(choices(data, weights=weights, k=20))
['blue', 'blue', 'blue', 'blue', 'green', 'green', 'green', 'green',
'green', 'green', 'green', 'green', 'red', 'red', 'red', 'red',
'red', 'red', 'red', 'red']

-tim






From arj.python at gmail.com  Fri Dec 28 08:27:09 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 28 Dec 2018 17:27:09 +0400
Subject: graded randomness
In-Reply-To: <875zvdx02k.fsf@bsb.me.uk>
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
 <mailman.44.1545999024.2779.python-list@python.org> <875zvdx02k.fsf@bsb.me.uk>
Message-ID: <CADrxXXkR4Mys5k8-bFG+X6-Jdn-xCSm1mUvtERyF-G-JYzxBMQ@mail.gmail.com>

woops @ben me too i got that solution but i'm searching for a neater answer.

let us say i'm using it over new data, 1 million data of red blue green can
be saved by just dealing with the probabilities ^^_

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius


From arj.python at gmail.com  Fri Dec 28 08:31:32 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 28 Dec 2018 17:31:32 +0400
Subject: graded randomness
In-Reply-To: <20181228072339.3f6dac35@bigbox.christie.dr>
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
 <20181228072339.3f6dac35@bigbox.christie.dr>
Message-ID: <CADrxXXk_ROiophHQSPZUkc3NChr-=9A7B-ovcq+p_HZfL9HO0g@mail.gmail.com>

@Tim

do you have something like

choice(balls)

>>> red

and subsequent repetitions for long enough yield approximately 2/5 times r
2/5 times g and 1/5 b

like one without random choice over list/tuples

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius


From rosuav at gmail.com  Fri Dec 28 08:32:42 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 29 Dec 2018 00:32:42 +1100
Subject: graded randomness
In-Reply-To: <CADrxXXkR4Mys5k8-bFG+X6-Jdn-xCSm1mUvtERyF-G-JYzxBMQ@mail.gmail.com>
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
 <mailman.44.1545999024.2779.python-list@python.org> <875zvdx02k.fsf@bsb.me.uk>
 <CADrxXXkR4Mys5k8-bFG+X6-Jdn-xCSm1mUvtERyF-G-JYzxBMQ@mail.gmail.com>
Message-ID: <CAPTjJmpsaoKwhxFrB8nMKvxau1E2aJnUA0v-Kptg4n9aZ4njDg@mail.gmail.com>

On Sat, Dec 29, 2018 at 12:28 AM Abdur-Rahmaan Janhangeer
<arj.python at gmail.com> wrote:
>
> woops @ben me too i got that solution but i'm searching for a neater answer.
>
> let us say i'm using it over new data, 1 million data of red blue green can
> be saved by just dealing with the probabilities ^^_

Assuming you're on Python 3.6 or newer, you should be able to use
random.choices() for this.

https://docs.python.org/3/library/random.html#random.choices

>>> random.choices("spam", [10, 1, 3, 2])
['a']

You can get any of the four letters back, but you're more likely to
get 's' than anything else. Is that the sort of thing you're after?

ChrisA


From arj.python at gmail.com  Fri Dec 28 08:38:19 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 28 Dec 2018 17:38:19 +0400
Subject: graded randomness
In-Reply-To: <CAPTjJmpsaoKwhxFrB8nMKvxau1E2aJnUA0v-Kptg4n9aZ4njDg@mail.gmail.com>
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
 <mailman.44.1545999024.2779.python-list@python.org> <875zvdx02k.fsf@bsb.me.uk>
 <CADrxXXkR4Mys5k8-bFG+X6-Jdn-xCSm1mUvtERyF-G-JYzxBMQ@mail.gmail.com>
 <CAPTjJmpsaoKwhxFrB8nMKvxau1E2aJnUA0v-Kptg4n9aZ4njDg@mail.gmail.com>
Message-ID: <CADrxXXnZK+Btmi=GBYtf2m7pBkZJuKDnbw-hZDtWk7Br42Oodw@mail.gmail.com>

ah yes, powerful enough for further customisations

* scratches head

thanks everybody !

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius


From python.list at tim.thechases.com  Fri Dec 28 08:45:43 2018
From: python.list at tim.thechases.com (Tim Chase)
Date: Fri, 28 Dec 2018 07:45:43 -0600
Subject: graded randomness
In-Reply-To: <CADrxXXk_ROiophHQSPZUkc3NChr-=9A7B-ovcq+p_HZfL9HO0g@mail.gmail.com>
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
 <20181228072339.3f6dac35@bigbox.christie.dr>
 <CADrxXXk_ROiophHQSPZUkc3NChr-=9A7B-ovcq+p_HZfL9HO0g@mail.gmail.com>
Message-ID: <20181228074501.440ef6b5@bigbox.christie.dr>

On 2018-12-28 17:31, Abdur-Rahmaan Janhangeer wrote:
> do you have something like
> 
> choice(balls)
> 
> >>> red  

Don't specify the "k=" (which defaults to 1 if you omit it) and use
the first element of the results:

>>> from random import choices
>>> distribution = {"green":2, "red": 2, "blue": 1}
>>> data, weights = zip(*distribution.items())
>>> choices(data, weights)[0]
'red'


> and subsequent repetitions for long enough yield approximately 2/5
> times r 2/5 times g and 1/5 b

You can sample it yourself:

>>> from collections import defaultdict
>>> a = defaultdict(int)
>>> for i in range(10000):
...     a[choices(data, weights=weights)[0]] += 1
... 
>>> dict(a)
{'green': 3979, 'red': 4008, 'blue': 2013}

though if you plan to, then it might be better/faster to use
cum_weights instead, calculating it once and then reusing it rather
than having choices() re-calculate the cumulative-weights on every
call.


> like one without random choice over list/tuples

Not sure what you mean by this.

-tkc




From grant.b.edwards at gmail.com  Fri Dec 28 12:21:07 2018
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 28 Dec 2018 17:21:07 +0000 (UTC)
Subject: Undocumented issue: Open system call blocks on named pipes (and
 a feature request)
References: <SN6PR01MB46874663F4911125B1B0E6B5BDB60@SN6PR01MB4687.prod.exchangelabs.com>
Message-ID: <q05m22$7aa$1@blaine.gmane.org>

On 2018-12-27, Daniel Ojalvo via Python-list <python-list at python.org> wrote:

>>>> open("this_is_a_pipe")
><blocks>

Opening a tty device can also block[1].  However, if somebody is using
the open() builtin on tty devices that's probably the least of their
problems.

[1] Technically, opening any character-mode device could block --
    serial ports are the only "common" example I can think of.

-- 
Grant Edwards               grant.b.edwards        Yow! How many retured
                                  at               bricklayers from FLORIDA
                              gmail.com            are out purchasing PENCIL
                                                   SHARPENERS right NOW??



From rosuav at gmail.com  Fri Dec 28 12:40:26 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 29 Dec 2018 04:40:26 +1100
Subject: Undocumented issue: Open system call blocks on named pipes (and a
 feature request)
In-Reply-To: <q05m22$7aa$1@blaine.gmane.org>
References: <SN6PR01MB46874663F4911125B1B0E6B5BDB60@SN6PR01MB4687.prod.exchangelabs.com>
 <q05m22$7aa$1@blaine.gmane.org>
Message-ID: <CAPTjJmr5d-O9JJzFWQFQoO5KUDr-uhb4yG_ftr4+NHQdQtG_dA@mail.gmail.com>

On Sat, Dec 29, 2018 at 4:24 AM Grant Edwards <grant.b.edwards at gmail.com> wrote:
>
> On 2018-12-27, Daniel Ojalvo via Python-list <python-list at python.org> wrote:
>
> >>>> open("this_is_a_pipe")
> ><blocks>
>
> Opening a tty device can also block[1].  However, if somebody is using
> the open() builtin on tty devices that's probably the least of their
> problems.

Depending on what you mean by "block", pretty much anything can. It's
not indefinite, but this is clearly an example of blocking behaviour:

rosuav at sikorsky:~$ mkdir gideon_home
rosuav at sikorsky:~$ sshfs gideon: gideon_home/
rosuav at sikorsky:~$ python3
Python 3.8.0a0 (heads/master:8b9c33ea9c, Nov 20 2018, 02:18:50)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.time(); open("gideon_home/.bashrc"); time.time()
1546018477.1130645
<_io.TextIOWrapper name='gideon_home/.bashrc' mode='r' encoding='UTF-8'>
1546018477.8339248

Due to the latency introduced by having a completely-out-of-cache
remote access directory, simply opening an ordinary file with an
ordinary path took over half a second. But *this* type of blocking
access is NOT changed by adding os.O_NONBLOCK; it will still take that
half second even if you say "opener=nonblock". OTOH, this form of
blocking is a lot safer - normal file systems [1] might be slow, but
they can't deadlock, whereas a pipe most certainly could.

ChrisA

[1] Of course, you could use fusermount and shenanigans to do
basically anything. But at that point, you're deliberately shooting
yourself in the foot, and all I can advise is "don't do that".


From avigross at verizon.net  Fri Dec 28 13:13:51 2018
From: avigross at verizon.net (Avi Gross)
Date: Fri, 28 Dec 2018 13:13:51 -0500
Subject: dangerous class neighborhood
In-Reply-To: <CADrxXXkzAkKewhLZsArED7mp_TQVBeWFz6J2Qc8kLi1dUaMJZQ@mail.gmail.com>
References: <005f01d49e2d$8ed49380$ac7dba80$@verizon.net>
 <CADrxXXknL5qZnMVyPr4Tx4SoSsR+Lf4_pc2bCXhS9Pt7ZaxQHw@mail.gmail.com>
 <010201d49e7b$1b747510$525d5f30$@verizon.net>
 <CADrxXXkzAkKewhLZsArED7mp_TQVBeWFz6J2Qc8kLi1dUaMJZQ@mail.gmail.com>
Message-ID: <006601d49ed9$16236fe0$426a4fa0$@verizon.net>

[REAL SUBJECT: an assortment of nothings]

 

Python tricks?

 

Speaking from a tutorial forum, tricks can catch people trying to learn.

 

I am not sure of the history of the ellipsis object. 

 

>>> dir(...)

['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']

 

Would you believe it is syntactically valid to use 4 periods in a row by asking for the contents like this:

 

>>> ....__class__

<class 'ellipsis'>

 

My impression is that the main initial use for the ellipsis was as a placeholder in a numpy multidimensional array so you can specify which axis you want to get all of:

 

https://python-reference.readthedocs.io/en/latest/docs/brackets/ellipsis.html

 

I don?t want to start yet another discussion but there seem to be a range of ways to say you want something without quite saying what you want. We have mentioned the ?pass? argument. We also have the naked colon as in:

 

new = old[ : ]

 

The above will make a shallow copy of a list. It is not really a new symbol but what happens when you take from:to and remove the from and the to and allow the defaults to take over. It gets murky as often underneath it all there are hidden objects like the slice:

 

>>> alpha = list(range(10))

>>> alpha

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

 

So you can explicitly index with a slice object or use the colon notation to get the same thing:

 

>>> alpha[1:5]

[1, 2, 3, 4]

>>> alpha[slice(1,5)]

[1, 2, 3, 4]

>>> alpha[1:10:2]

[1, 3, 5, 7, 9]

>>> alpha[slice(1,10,2)]

[1, 3, 5, 7, 9]

>>> alpha[5:]

[5, 6, 7, 8, 9]

>>> alpha[slice(5,)]

[0, 1, 2, 3, 4]

>>> alpha[slice(5,None)]

[5, 6, 7, 8, 9]

>>> alpha[:]

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> alpha[slice(,)]

SyntaxError: invalid syntax

>>> alpha[slice(None)]

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

 

Note the placeholder is now the item called ?None?

 

The ellipsis can be used here but causes errors.

 

And there are more subtle placeholders with meanings like DOES not exist. Numpy has a ?nan? and Pandas has NaN and a missing date is NaT and beneath it all there are names for a missing int versus float versus who knows what.

 

These can be subtle ideas for a newcomer, even for people who come from languages that don?t have the distinctions. For that matter, there are ways to say something is infinite which is not the same as saying the maximum value you can store in N bits.

 

But I express some caution on calling some of these things tricks. Yes, of course they can be tricky. What they often are may well be considered additional functionality by way of abstractions. Clearly a key word like ?pass? may be easier to understand than some of the others.

 

 

From: Abdur-Rahmaan Janhangeer <arj.python at gmail.com> 
Sent: Friday, December 28, 2018 3:46 AM
To: Avi Gross <avigross at verizon.net>
Cc: Python <python-list at python.org>
Subject: Re: dangerous class neighborhood

 

thanks, the 3 dots are more explicit, thought was another py trick

 

yours,

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius



From arj.python at gmail.com  Fri Dec 28 14:44:39 2018
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 28 Dec 2018 23:44:39 +0400
Subject: graded randomness
In-Reply-To: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
Message-ID: <CADrxXXkjqJRwowGv49EKh6BaN4pDWTzKB8RngCZey=KiNgXsjA@mail.gmail.com>

well i wanted that to improve the following code:

https://www.pythonmembers.club/2018/12/28/reviving-bertrand-russell-through-python/

that one i used the random list technique

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius

>


From D.Ojalvo at F5.com  Fri Dec 28 15:21:05 2018
From: D.Ojalvo at F5.com (Daniel Ojalvo)
Date: Fri, 28 Dec 2018 20:21:05 +0000
Subject: Undocumented issue: Open system call blocks on named pipes (and a
 feature request)
In-Reply-To: <CAPTjJmrFbEee5joZo2_h1_vOD=Ts1uzusJB4SMWJXYU1BJTYTQ@mail.gmail.com>
References: <SN6PR01MB46874663F4911125B1B0E6B5BDB60@SN6PR01MB4687.prod.exchangelabs.com>
 <CAPTjJmrFbEee5joZo2_h1_vOD=Ts1uzusJB4SMWJXYU1BJTYTQ@mail.gmail.com>
Message-ID: <SN6PR01MB4687680F3BA48F90634887B0BDB70@SN6PR01MB4687.prod.exchangelabs.com>

Thank you for the advice!

I haven't used the opener argument before, but I'll keep it for future reference. I think it's still a little kludge-y, but it works.

I agree that previous behavior shouldn't be changed, but I would suggest updating the documentation to point it out as a footnote. The current behavior is correct just unclear. Most people just learning about the open command wouldn't have this expectation. I came across the issue when I had a program that would open up all the files in a directory to read a few bytes from the beginning. My concern would be someone just making a named pipe over a file that a program would open. Arguably, anyone affected by that would be shooting themselves in the foot to begin with, but I think there are "security" concerns because someone could cause a bit of mischief that would be difficult to diagnose.

That all being said, I think I would like to put in a feature request for a non-blocking option. How should I go about doing so?

Thanks again,
Dan

-----Original Message-----
From: Chris Angelico <rosuav at gmail.com> 
Sent: Thursday, December 27, 2018 7:10 PM
To: python-list at python.org
Subject: Re: Undocumented issue: Open system call blocks on named pipes (and a feature request)

On Fri, Dec 28, 2018 at 1:38 PM Daniel Ojalvo via Python-list <python-list at python.org> wrote:
>
> Hello,
>
> I've been working on a python3 project and I came across an issue with the open system call that, at the very least, isn't documented. In my humble opinion, the documentation<https://docs.python.org/3/library/functions.html#open> should be updated because folks wouldn't expect open to be a blocking operation and simply error out. Worse yet, open doesn't have an option to make itself non-blocking. You have to use the os system calls to kludge a solution.
>

Hmm. I disagree that the docs are deceptive here; I would normally expect open() to block if it needs to. But looking at this as a feature request, it seems reasonable. Actually, it's not even that hard to do, since open() is already pluggable:

rosuav at sikorsky:~/tmp$ rm rene_magritte
rosuav at sikorsky:~/tmp$ mkfifo rene_magritte rosuav at sikorsky:~/tmp$ ls -l rene_magritte
prw-r--r-- 1 rosuav rosuav 0 Dec 28 14:05 rene_magritte rosuav at sikorsky:~/tmp$ python3 Python 3.8.0a0 (heads/master:8b9c33ea9c, Nov 20 2018, 02:18:50) [GCC 6.3.0 20170516] on linux Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> def nonblock(fn, mode): return os.open(fn, mode | os.O_NONBLOCK)
...
>>> open("rene_magritte", opener=nonblock)
<_io.TextIOWrapper name='rene_magritte' mode='r' encoding='UTF-8'>
>>> _.read(1)
''

> Here is how I reproduced the issue:
>
> root at beefy:~/sandbox# mkfifo this_is_a_pipe

(my example file name is a more subtle reference...)

> I'm doing this to get a fileobject and make it error out if we do have a blocking special file:
> with os.fdopen(os.open(<filename>, os.O_RDONLY| os.O_NONBLOCK) , mode='rb') as file_obj:
>
> I think this is mostly a documentation bug because this wouldn't be expected behavior to someone reading the docs, but open is behaving as the fifo man page<http://man7.org/linux/man-pages/man7/fifo.7.html> is documented. The feature request would be to add a non-blocking option to the default open system call.
>

Honestly, I don't think there's a problem with it blocking by default.
Most of Python works that way. But it would be pretty straight-forward to add "nonblocking=False" as another keyword-only parameter, and for compatibility with existing versions (back as far as 3.3), the opener should work just fine.

ChrisA


From rosuav at gmail.com  Fri Dec 28 16:04:17 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 29 Dec 2018 08:04:17 +1100
Subject: Undocumented issue: Open system call blocks on named pipes (and a
 feature request)
In-Reply-To: <SN6PR01MB4687680F3BA48F90634887B0BDB70@SN6PR01MB4687.prod.exchangelabs.com>
References: <SN6PR01MB46874663F4911125B1B0E6B5BDB60@SN6PR01MB4687.prod.exchangelabs.com>
 <CAPTjJmrFbEee5joZo2_h1_vOD=Ts1uzusJB4SMWJXYU1BJTYTQ@mail.gmail.com>
 <SN6PR01MB4687680F3BA48F90634887B0BDB70@SN6PR01MB4687.prod.exchangelabs.com>
Message-ID: <CAPTjJmrTw4_H9pEhbw+baguAohW6a-bVW5fiWJNQEW0L=eLRTA@mail.gmail.com>

On Sat, Dec 29, 2018 at 7:21 AM Daniel Ojalvo <D.Ojalvo at f5.com> wrote:
>
> Thank you for the advice!
>
> I haven't used the opener argument before, but I'll keep it for future reference. I think it's still a little kludge-y, but it works.

It has a very similar effect to what you were proposing, but still
works within the normal open() ecosystem. Its main benefit is that it
works on existing Pythons, whereas any idea proposed today can't get
into 3.7 and maybe not even 3.8.

> I agree that previous behavior shouldn't be changed, but I would suggest updating the documentation to point it out as a footnote. The current behavior is correct just unclear. Most people just learning about the open command wouldn't have this expectation.
>

That's what I'm not sure about. Do people really have an expectation
of nonblocking behaviour?

> I came across the issue when I had a program that would open up all the files in a directory to read a few bytes from the beginning. My concern would be someone just making a named pipe over a file that a program would open. Arguably, anyone affected by that would be shooting themselves in the foot to begin with, but I think there are "security" concerns because someone could cause a bit of mischief that would be difficult to diagnose.
>

What happens if someone has a subdirectory in there? To be resilient
against everything you might come across, you probably need to check
anyway.

> That all being said, I think I would like to put in a feature request for a non-blocking option. How should I go about doing so?

Hmm, there are a few options. If you reckon it's pretty easy, you
could just go straight to a GitHub pull request, and discuss it there.
Or you could open a bugs.python.org tracker issue and hope someone
else does the coding. Alternatively, you could find out who else
supports the idea by posting to the python-ideas at python.org mailing
list.

Of those, I think posting to python-ideas is possibly the best, as
there will likely be some bikeshedding about the name and such.

ChrisA


From avigross at verizon.net  Fri Dec 28 18:01:57 2018
From: avigross at verizon.net (Avi Gross)
Date: Fri, 28 Dec 2018 18:01:57 -0500
Subject: graded randomness
In-Reply-To: <CADrxXXkjqJRwowGv49EKh6BaN4pDWTzKB8RngCZey=KiNgXsjA@mail.gmail.com>
References: <CADrxXXmNu5F8J=N_doYL2Bh=kw_dJD4pOecc6Y6vW+cB_3HnxQ@mail.gmail.com>
 <CADrxXXkjqJRwowGv49EKh6BaN4pDWTzKB8RngCZey=KiNgXsjA@mail.gmail.com>
Message-ID: <00af01d49f01$558e2670$00aa7350$@verizon.net>

Abdur-Rahman

I am sure various modules available have ready-made solutions and I see
others have replied to your question. The usual disclaimers apply. This is
an academic discussion and not a statement of the right or only way to do an
abstract task.

So just a thought. You seem interested in a GENERAL situation where you have
N situations with each having a specific probability and the probabilities
sum to 1.0. If that is right, you are creating a partition where you can
make a data structure listing N ordered items along with their individual
probability. Given such a list, you can create another item that is a
cumulative sum.

In your example, your individual probabilities for ['green', 'red', 'blue']
are [0.4, 0.4, 0.2] but the use of lists is just for illustration. You might
use a Numpy array as they have a cumulative sum function:

import numpy as np
np.cumsum([0.4, 0.4, 0.2])
Returns:
array([0.4, 0.8, 1. ])

or viewed vertically:

np.cumsum([0.4, 0.4, 0.2]).reshape(3,1)
array([[0.4],
       [0.8],
       [1. ]])

Again, we are talking about a GENERAL solution but using this example to
illustrate. To get a weighted probability now, you use the random module (or
anything else) to generate a random number between 0 and 1. You search in
the cumulative sum data structure to find the right range. A random value
less than 0.4 should direct you to using green. If above that but less than
0.8, use red. Else, use blue.

To do this properly, you can decide what data structures makes this easy to
do. Maintaining three independent lists or arrays may not be optimal. 

The main idea is to find a way to segment your choices. So consider a
different common example of rolling a pair of (six sided) standard dice. You
know there are 6**2 possible outcomes. There is only one way to get a sum of
2 by rolling a one and another one. So the probability is 1/36 or .0277...
and you can calculate the probabilities of all the other sums with a 7 being
the most common roll at .1667. In this example your choices are rolls of 2
through 12 or 11 choices. The same logic applies. Generate 11 data measures
and a cumulative sum. Just as illustration, I show code using a Pandas
DataFrame object.

import numpy as np
import pandas as pd
diceSumText = np.array(["two", "three", "four", "five", "six", "seven",
"eight", "nine", "ten", "eleven", "twelve"])
diceSumVal = np.array(range(2,13))
diceProbability = np.array([1, 2, 3, 4, 5, 6, 5, 4, 3, 2, 1]) / 36
diceProbCumSum = np.cumsum(diceProbability)

Now combine those:

mydata = pd.DataFrame({"Text": diceSumText, "Value": diceSumVal, "Prob":
diceProbability, "Cum": diceProbCumSum})
print(mydata)

      Text  Value      Prob       Cum
0      two      2  0.027778  0.027778
1    three      3  0.055556  0.083333
2     four      4  0.083333  0.166667
3     five      5  0.111111  0.277778
4      six      6  0.138889  0.416667
5    seven      7  0.166667  0.583333
6    eight      8  0.138889  0.722222
7     nine      9  0.111111  0.833333
8      ten     10  0.083333  0.916667
9   eleven     11  0.055556  0.972222
10  twelve     12  0.027778  1.000000

Again, you can do something any number of ways. This is just one. And in
this format the indentation is not great. 

But it lets you write an algorithm that finds the highest 'index" that still
is below the random number chosen and then select either the text or value
that fits in that partition. Not to repeat, there are many other ways so
feel free to innovate. 


-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Abdur-Rahmaan Janhangeer
Sent: Friday, December 28, 2018 2:45 PM
To: Python <python-list at python.org>
Subject: Re: graded randomness

well i wanted that to improve the following code:

https://www.pythonmembers.club/2018/12/28/reviving-bertrand-russell-through-
python/

that one i used the random list technique

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ Mauritius

>
--
https://mail.python.org/mailman/listinfo/python-list



From cs at cskk.id.au  Fri Dec 28 18:37:14 2018
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 29 Dec 2018 10:37:14 +1100
Subject: Undocumented issue: Open system call blocks on named pipes (and
 a feature request)
In-Reply-To: <SN6PR01MB4687680F3BA48F90634887B0BDB70@SN6PR01MB4687.prod.exchangelabs.com>
References: <SN6PR01MB4687680F3BA48F90634887B0BDB70@SN6PR01MB4687.prod.exchangelabs.com>
Message-ID: <20181228233714.GA24742@cskk.homeip.net>

On 28Dec2018 20:21, Daniel Ojalvo <D.Ojalvo at F5.com> wrote:
>I agree that previous behavior shouldn't be changed, but I would 
>suggest updating the documentation to point it out as a footnote. The 
>current behavior is correct just unclear. Most people just learning 
>about the open command wouldn't have this expectation.

Maybe, maybe not. "Most" is a conjecture. IMO people will only find it 
surprising if they think any filesystem object can be instantly opened.  
However that is a misapprehension on their part.

My personal expectation is that open() will come back when the object is 
open. I don't have a timeframe in mind unless I have a strong 
expectation about _what_ I'm opening.

>I came across the issue when I had a program that would open up all the 
>files in a directory to read a few bytes from the beginning. My concern 
>would be someone just making a named pipe over a file that a program 
>would open.

What about a symlink to a magic /dev/tcp/host:port device, initiating a 
TCP connection? Particularly if "host" is down or inaccessible? Etc.

>Arguably, anyone affected by that would be shooting themselves in the 
>foot to begin with, but I think there are "security" concerns because 
>someone could cause a bit of mischief that would be difficult to 
>diagnose.

It isn't hard to diagnose at all. Point strace at the hung pogram, see 
it is opening some path, "ls -ld the-path", oooh, it isn't a regular 
file.

The point here is that if a programme opens every file in a directory, 
maybe it should constrain itself to regular files. Opening anything else 
may not just hang, it can have real world side effects. (Usually such 
effect happen at some point after open, for example opening a rewind 
take device will physicially rewind the tape on close, but you've 
committed to that happening by opening it in the first place.)

I think Chris offered the example of a subdirectory to suggest that such 
a programme already has an opnion about what to open and what to leave 
alone (unless is _does_ open() subdirectories, which might be useful but 
is usually misleading and on some OSes unsupported). So the programme 
should be pickier anyway.

>That all being said, I think I would like to put in a feature request 
>for a non-blocking option. How should I go about doing so?

I agree with the suggestion already made: devise a well thought out 
proposal which fits nicely with the existing open() call (eg an addition 
to the mode argument or something), and describe it clearly in 
python-ideas.

Certainly a number of things can be opened in a "nonblocking" mode, 
which means that reads return instantly if there's no available data, so 
having an open not block isn't unreasonable to want. But it may be 
unreasonable to implement in general: OSes may not support it directly.

Cheers,
Cameron Simpson <cs at cskk.id.au>


From jfong at ms4.hinet.net  Fri Dec 28 21:39:29 2018
From: jfong at ms4.hinet.net (jfong at ms4.hinet.net)
Date: Fri, 28 Dec 2018 18:39:29 -0800 (PST)
Subject: Ask for help about class variable scope (Re: Why doesn't a
 dictionary work in classes?)
In-Reply-To: <39b5b835-9960-4fca-a2ca-b07f72e3f6d2@googlegroups.com>
References: <69c781e6-a1b6-4d2c-9b76-3f7ea434f495@googlegroups.com>
 <CACL+1avwhLj5Q9ygM_VEV7veL-7-Yc7Ewb8nzYnknLVYMUW2Kw@mail.gmail.com>
 <mailman.25.1545892306.2779.python-list@python.org>
 <b5f17aec-ed07-4607-959e-bbe981e6dab0@googlegroups.com>
 <CACL+1auHhjpUMdsWQ2UuYt8nT2mre8TruEyWZEY4EGBUmDc8fA@mail.gmail.com>
 <mailman.27.1545908300.2779.python-list@python.org>
 <39b5b835-9960-4fca-a2ca-b07f72e3f6d2@googlegroups.com>
Message-ID: <607e003b-51c7-4ea4-9ab3-c536e41bbd31@googlegroups.com>

jf... at ms4.hinet.net? 2018?12?28???? UTC+8??4?04?07????
> eryk sun at 2018/12/27 UTC+8 PM 6:58:33 wrote:
> > On 12/27/18, jfong at ms4.hinet.net <jfong at ms4.hinet.net> wrote:
> > >
> > > I still don't get it. When I change it to using list comprehension, the
> > > problem is still there. (it now has no late-binding variable, right? :-)
> > >
> > >>>> class Too:
> > > ...     XS = [15, 15, 15, 15]
> > > ...     Z4 = [val for val in XS]
> > > ...     Z5 = [XS[0] for val in XS]
> > > ...
> > > Traceback (most recent call last):
> > >   File "<stdin>", line 1, in <module>
> > >   File "<stdin>", line 4, in Too
> > >   File "<stdin>", line 4, in <listcomp>
> > > NameError: name 'XS' is not defined
> > >
> > > The problem confuse me is that is XS a local variable of the list
> > > comprehension?
> > 
> > XS is not a local variable in the scope of either comprehension. XS is
> > local to the class statement's scope. For each comprehension, an
> > iterator for the current object referenced by XS gets instantiated
> > (early binding) and passed as an argument to the comprehension scope.
> > If we disassemble the comprehension code, we find that this iterator
> > argument has the creatively illegal name ".0". (The bytecode
> > references it by its fast-locals-array index, not its weird name.)
> > 
> > In the Z5 case, XS is a non-local variable (late binding) in the
> > loop-body expression (left-hand side) of the comprehension. That's
> > common for Python code. In every iteration of the loop, the
> > interpreter looks up the object referenced by the name "XS", which can
> > change at any time (e.g. by another thread).
> 
> In Python document 4.2.2. Resolution of names, the last paragraph:
> 
>     "...A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution with an exception that unbound local variables are looked up in the global namespace. ...The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods ? this includes comprehensions and generator expressions since they are implemented using a function scope". 
> 
> These statements reflect the following difference:
> 
> >>> xy = [1,2]
> >>> [dir() for i in xy]
> [['.0', 'i'], ['.0', 'i']]
> >>> [xy[0] for i in xy]
> [1, 1]
> 
> >>> class foo():
> ...     xs = [1,2]
> ...     z4 = [dir() for i in xs]
> ...
> >>> foo().z4
> [['.0', 'i'], ['.0', 'i']]
> >>> class foo():
> ...     xs = [1,2]
> ...     z4 = [xs[0] for i in xs]
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 3, in foo
>   File "<stdin>", line 3, in <listcomp>
> NameError: name 'xs' is not defined
> >>>
> 
> and it goes further:
> 
> >>> [dir() for i in xy for j in xy]
> [['.0', 'i', 'j'], ['.0', 'i', 'j'], ['.0', 'i', 'j'], ['.0', 'i', 'j']]
> >>> class foo():
> ...     xs = [1,2]
> ...     z5 = [dir() for i in xs for j in xs]
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 3, in foo
>   File "<stdin>", line 3, in <listcomp>
> NameError: name 'xs' is not defined
> >>>
> 
> That's all I had learn so far, although not understand the design decision behind it yet:-(
> 
> --Jach

Anyway, it "looks" weird. Isn't it?

>>> xs = [5,6,7,8]
>>> class foo():
...     xs = [1,2,3]
...     z4 = [xs[i] for i in xs]
...
>>> foo.z4
[6,7,8]

xs in the "for" statement referenced to class variable xs and xs[i] in the body referenced to the global xs with i from locals.

PS. Don't bother to reply. This is just for my own documentary:-)


From gisle.vanem at gmail.com  Fri Dec 28 23:16:41 2018
From: gisle.vanem at gmail.com (Gisle Vanem)
Date: Sat, 29 Dec 2018 05:16:41 +0100
Subject: Passing back the ERRORLEVEL
Message-ID: <75aaa440-74f6-ff91-23f0-2e0f0555b2b0@gmail.com>

I have trouble understanding why a 'sys.exit(2)' is
*not* passed back to the CMD shell in this little example:

----- c:\py_cmd_test.cmd ------
@%WinDir%\py.exe -2 -x %~dp0py_cmd_test.cmd & exit /b %ERRORLEVEL%

# The '-x' is for Python to skip the 1st line of this file.
import sys
print ("Hello, I am %s; %s" % (sys.argv[0], sys.version_info))
sys.exit (2)

--------

Executing this in a 4NT shell (from JPsoft), correctly reports:
c:\> py_cmd_test.cmd & echo %errorlevel
Hello, I am C:\py_cmd_test.cmd; sys.version_info(major=2, minor=7, micro=15, releaselevel='final', serial=0)
2

But the junk-ware cmd.exe does not:
C:\>py_cmd_test.cmd & echo %ERRORLEVEL%
Hello, I am C:\py_cmd_test.cmd; sys.version_info(major=2, minor=7, micro=15, releaselevel='final', serial=0)
0

Anybody here who can spot the problem?

Same issue with 'py -3'. I'm on Windows-10.

-- 
--gv


From jbl.rand at gmail.com  Sat Dec 29 03:15:53 2018
From: jbl.rand at gmail.com (Rand .J)
Date: Sat, 29 Dec 2018 11:15:53 +0300
Subject: Begginers problem : tar: Error opening archive: Can't initialize
 filter; unable to run program "bzip2 -d
Message-ID: <CAGjF-q3nOZ+MwvZ0jYYH6qZrsXFOVCi1LiJ5crptxn821mYK+g@mail.gmail.com>

Hello everyone

I'm completely new to python and I'm facing this problem :

I'm trying to run this code from : https://github.com/pnnl/*safekit* ,using
cmd on windows 10, I already installed python. when I type the command: tar
-xjvf data_examples.tar.bz2

I keep getting the error: tar: Error opening archive: Can't initialize
filter; unable to run program "bzip2 -d"

I have tried to download bzip2 through easy-7 zip and GnuWin32 , but it
didn't work. Can any one help me? Thank you

P.S.: this might not be a Python specific problem, but I searched for the
same problem before posting mine.


I apologize for any inconvenience

Thank you


From eryksun at gmail.com  Sat Dec 29 04:05:47 2018
From: eryksun at gmail.com (eryk sun)
Date: Sat, 29 Dec 2018 03:05:47 -0600
Subject: Passing back the ERRORLEVEL
In-Reply-To: <75aaa440-74f6-ff91-23f0-2e0f0555b2b0@gmail.com>
References: <75aaa440-74f6-ff91-23f0-2e0f0555b2b0@gmail.com>
Message-ID: <CACL+1auGzvCESuj31nW9Kpzz0m2bWfjGx0AK4-bsb81AVymWHg@mail.gmail.com>

On 12/28/18, Gisle Vanem <gisle.vanem at gmail.com> wrote:
> I have trouble understanding why a 'sys.exit(2)' is
> *not* passed back to the CMD shell in this little example:
>
> ----- c:\py_cmd_test.cmd ------
> @%WinDir%\py.exe -2 -x %~dp0py_cmd_test.cmd & exit /b %ERRORLEVEL%

%ERRORLEVEL% gets expanded in the initial parsing before this line is
executed, at which time it is 0. Because of the /b argument, you're
not actually exiting CMD and have no reason to change the error value.
Just use `exit /b`.

Why use CMD as a launcher when you have py.exe with shebang support?


From bhagvanarch at gmail.com  Sat Dec 29 06:17:44 2018
From: bhagvanarch at gmail.com (bhagvanarch at gmail.com)
Date: Sat, 29 Dec 2018 03:17:44 -0800 (PST)
Subject: Monte Carlo Quant: Opensource project
Message-ID: <f68b4a95-6368-456f-affa-1cda6639bf6a@googlegroups.com>

Quantica computacao has started an open source initiative in python. We invite students and professionals to be part of it . My email address is bhagvank at quanticacomputacao.com or bhagvanarch at gmail.com

Repository link is : https://github.com/bhagvank/Montecarlo_Quant
If you want to participate and get invitation, please share the github ids.



A slack channel for monte carlo open source:

https://goo.gl/iM29fH

Telegram Group for Monte Carlo Open source:

https://t.me/joinchat/KisuQBYtJbinYm5tBytOTw


From python at bdurham.com  Sat Dec 29 13:48:00 2018
From: python at bdurham.com (Malcolm Greene)
Date: Sat, 29 Dec 2018 11:48:00 -0700
Subject: Logging - have logger use caller's function name and line number
Message-ID: <1546109280.3533148.1620745608.64E15FB0@webmail.messagingengine.com>

I have a class method that adds additional context to many of the
class's log messages. Rather than calling, for example,  logger.info(
'message ...' ) I would like to call my_object.log( 'message ...' ) so
that this additional context is managed in a single place. I'm actually
doing that and its working great except that ... as expected ... all my
log records have the my_object.log method's name and line numbers vs the
callers' names and line numbers.
Is there a technique I can use to have my custom log method inject the
caller's name and line numbers in the log output?


From __peter__ at web.de  Sat Dec 29 15:45:44 2018
From: __peter__ at web.de (Peter Otten)
Date: Sat, 29 Dec 2018 21:45:44 +0100
Subject: Logging - have logger use caller's function name and line number
References: <1546109280.3533148.1620745608.64E15FB0@webmail.messagingengine.com>
Message-ID: <q08m9l$5h1$1@blaine.gmane.org>

Malcolm Greene wrote:

> I have a class method that adds additional context to many of the
> class's log messages. Rather than calling, for example,  logger.info(
> 'message ...' ) I would like to call my_object.log( 'message ...' ) so
> that this additional context is managed in a single place. I'm actually
> doing that and its working great except that ... as expected ... all my
> log records have the my_object.log method's name and line numbers vs the
> callers' names and line numbers.
> Is there a technique I can use to have my custom log method inject the
> caller's name and line numbers in the log output?

https://mail.python.org/pipermail/python-list/2010-March/570941.html

No idea if this works with current Python, or if there is a more elegant way 
by now.



From bouncingcats at gmail.com  Sat Dec 29 18:36:16 2018
From: bouncingcats at gmail.com (David)
Date: Sun, 30 Dec 2018 10:36:16 +1100
Subject: Logging - have logger use caller's function name and line number
In-Reply-To: <1546109280.3533148.1620745608.64E15FB0@webmail.messagingengine.com>
References: <1546109280.3533148.1620745608.64E15FB0@webmail.messagingengine.com>
Message-ID: <CAMPXz=peCgj40baQdja20f+eVzppTFUsk9VDN8dXVpfW9ogtVg@mail.gmail.com>

On Sun, 30 Dec 2018 at 05:58, Malcolm Greene <python at bdurham.com> wrote:
>
> I have a class method that adds additional context to many of the
> class's log messages. Rather than calling, for example,  logger.info(
> 'message ...' ) I would like to call my_object.log( 'message ...' ) so
> that this additional context is managed in a single place. I'm actually
> doing that and its working great except that ... as expected ... all my
> log records have the my_object.log method's name and line numbers vs the
> callers' names and line numbers.
> Is there a technique I can use to have my custom log method inject the
> caller's name and line numbers in the log output?

You can use this:
https://docs.python.org/3/library/inspect.html#the-interpreter-stack


From torriem at gmail.com  Sat Dec 29 20:22:10 2018
From: torriem at gmail.com (Michael Torrie)
Date: Sat, 29 Dec 2018 18:22:10 -0700
Subject: Begginers problem : tar: Error opening archive: Can't initialize
 filter; unable to run program "bzip2 -d
In-Reply-To: <CAGjF-q3nOZ+MwvZ0jYYH6qZrsXFOVCi1LiJ5crptxn821mYK+g@mail.gmail.com>
References: <CAGjF-q3nOZ+MwvZ0jYYH6qZrsXFOVCi1LiJ5crptxn821mYK+g@mail.gmail.com>
Message-ID: <dbf00934-515d-c5f2-ea25-b3f24d5b9aff@gmail.com>

On 12/29/2018 01:15 AM, Rand .J wrote:
> I'm trying to run this code from : https://github.com/pnnl/*safekit* ,using
> cmd on windows 10, I already installed python. when I type the command: tar
> -xjvf data_examples.tar.bz2
> 
> I keep getting the error: tar: Error opening archive: Can't initialize
> filter; unable to run program "bzip2 -d"

It means that tar is unable to run bzip2 as a child process. Most likely
it simply cannot find it.
> I have tried to download bzip2 through easy-7 zip and GnuWin32 , but it
> didn't work. Can any one help me? Thank you

Is bzip2 in the path? Can you run it from the command line by itself?
Try just typing this at cmd: bzip2 --help

If that won't work, then you need to put bzip2 in the system PATH so tar
can find it.

Normally I run tar and friends via either cygwin or MingW's MSYS
environment where tar and bzip2 are installed from packages and are
already in the right place.




From cs at cskk.id.au  Sat Dec 29 20:29:39 2018
From: cs at cskk.id.au (Cameron Simpson)
Date: Sun, 30 Dec 2018 12:29:39 +1100
Subject: Begginers problem : tar: Error opening archive: Can't initialize
 filter; unable to run program "bzip2 -d
In-Reply-To: <CAGjF-q3nOZ+MwvZ0jYYH6qZrsXFOVCi1LiJ5crptxn821mYK+g@mail.gmail.com>
References: <CAGjF-q3nOZ+MwvZ0jYYH6qZrsXFOVCi1LiJ5crptxn821mYK+g@mail.gmail.com>
Message-ID: <20181230012939.GA23911@cskk.homeip.net>

On 29Dec2018 11:15, Rand .J <jbl.rand at gmail.com> wrote:
>I'm trying to run this code from : https://github.com/pnnl/*safekit* 
>,using cmd on windows 10, I already installed python. when I type the command: tar
>-xjvf data_examples.tar.bz2
>
>I keep getting the error: tar: Error opening archive: Can't initialize
>filter; unable to run program "bzip2 -d"
>
>I have tried to download bzip2 through easy-7 zip and GnuWin32 , but it
>didn't work. Can any one help me? Thank you
>
>P.S.: this might not be a Python specific problem, but I searched for the
>same problem before posting mine.

This isn't a Python issue: you're not even running Python.

GNU tar's decompress modes work by invoking an external programme to do 
the decompression. For the -j option, that programme is "bzip2 -d".

Where did you get tar from? Can you get bzip2 from the same place?  
Note: bzip2 is NOT related to the "zip" programme.

Personally, when I need to use Windows I like to install Cygwin:

  https://cygwin.com/

and pretend from then on that it is UNIX as far as I can.

Cygwin's installer has a package selection stage; you should be able to 
find bzip2, probably in some kind of "utilities" section. Grab Cygwin's 
tar package as well.

Cheers,
Cameron Simpson <cs at cskk.id.au>


From bhagvanarch at gmail.com  Sun Dec 30 06:58:07 2018
From: bhagvanarch at gmail.com (bhagvanarch at gmail.com)
Date: Sun, 30 Dec 2018 03:58:07 -0800 (PST)
Subject: Slack NLP : open source project
Message-ID: <4a57d4ba-9d5a-4142-b2da-fe9a07af8d27@googlegroups.com>

If you would like to join the slack NLP project, please share your github id to email address is bhagvank at quanticacomputacao.com or bhagvank at gmail.com.


Slack NLP github repository

https://github.com/bhagvank/slacknlp



Slack channel

http://goo.gl/BMGHjr



Telegram group

https://t.me/joinchat/KisuQBRcgs9F7x6lQESpmg



check out my latest presentation on open source project-SlackNLP at Pycon 2018

https://goo.gl/hpWu12


From cseberino at gmail.com  Sun Dec 30 21:49:23 2018
From: cseberino at gmail.com (Christian Seberino)
Date: Sun, 30 Dec 2018 18:49:23 -0800 (PST)
Subject: Question about slight deviations when using integer division with
 large integers.
Message-ID: <9618b456-e9c0-486d-977b-a45e7f694f14@googlegroups.com>

Why are the following two similar prints slightly different and how fix?

>>> x = 0x739ad43ed636

>>> print(x + (-x) // 2048)
127046758190683

>>> print(x - x // 2048)
127046758190684

I'm working in an area where such deviations matter.  It would nice to understand what is happening.  

Any help greatly appreciated.

cs


From cseberino at gmail.com  Sun Dec 30 21:54:49 2018
From: cseberino at gmail.com (Christian Seberino)
Date: Sun, 30 Dec 2018 18:54:49 -0800 (PST)
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <9618b456-e9c0-486d-977b-a45e7f694f14@googlegroups.com>
References: <9618b456-e9c0-486d-977b-a45e7f694f14@googlegroups.com>
Message-ID: <890c5fcc-b5a9-4ba3-a8d5-959ad3f19e5b@googlegroups.com>

Perhaps the "secret" is *not* do integer division with negative numbers?


From rosuav at gmail.com  Sun Dec 30 21:59:01 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 31 Dec 2018 13:59:01 +1100
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <890c5fcc-b5a9-4ba3-a8d5-959ad3f19e5b@googlegroups.com>
References: <9618b456-e9c0-486d-977b-a45e7f694f14@googlegroups.com>
 <890c5fcc-b5a9-4ba3-a8d5-959ad3f19e5b@googlegroups.com>
Message-ID: <CAPTjJmqX4E8DeeaPjW+k1vG9nJNmznYxKoJRxW7gfMsPuJ8hRw@mail.gmail.com>

On Mon, Dec 31, 2018 at 1:56 PM Christian Seberino <cseberino at gmail.com> wrote:
>
> Perhaps the "secret" is *not* do integer division with negative numbers?

I have no idea what you're replying to, but integer division with
negative numbers IS well defined. Python will floor - it will always
round down.

ChrisA


From cseberino at gmail.com  Mon Dec 31 00:14:04 2018
From: cseberino at gmail.com (Christian Seberino)
Date: Sun, 30 Dec 2018 21:14:04 -0800 (PST)
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <mailman.1.1546225155.4816.python-list@python.org>
References: <9618b456-e9c0-486d-977b-a45e7f694f14@googlegroups.com>
 <890c5fcc-b5a9-4ba3-a8d5-959ad3f19e5b@googlegroups.com>
 <CAPTjJmqX4E8DeeaPjW+k1vG9nJNmznYxKoJRxW7gfMsPuJ8hRw@mail.gmail.com>
 <mailman.1.1546225155.4816.python-list@python.org>
Message-ID: <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>

What is simplest way to make both those 
prints give same values?  Any slicker way 
than an if statement?


From cs at cskk.id.au  Mon Dec 31 00:24:39 2018
From: cs at cskk.id.au (Cameron Simpson)
Date: Mon, 31 Dec 2018 16:24:39 +1100
Subject: Question about slight deviations when using integer division
 with large integers.
In-Reply-To: <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>
References: <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>
Message-ID: <20181231052439.GA2726@cskk.homeip.net>

On 30Dec2018 21:14, Christian Seberino <cseberino at gmail.com> wrote:
>What is simplest way to make both those
>prints give same values?  Any slicker way
>than an if statement?

If your post had an attachment, be aware that the python-list list drops 
all attachments - it is a text only list. Please paste your code 
directly into your messages so that others can easily see it.

Thanks,
Cameron Simpson <cs at cskk.id.au>


From cseberino at gmail.com  Mon Dec 31 00:33:59 2018
From: cseberino at gmail.com (Christian Seberino)
Date: Sun, 30 Dec 2018 23:33:59 -0600
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <20181231052439.GA2726@cskk.homeip.net>
References: <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>
 <20181231052439.GA2726@cskk.homeip.net>
Message-ID: <CAG5-5iKJQ=CixrK9ce4fbSeV1uXvwJvEaCHDRE-wB+eOHHOisw@mail.gmail.com>

Thanks.  I didn?t post new code.  I was just referring back to original
post.  I need to duplicate the exact behavior of Java?s BigIntegers.

I?m guessing difference between Java and Python is that Java BigIntegers do
not switch to floor for negatives.

Possible to tweak rounding of Python to be like Java?

On Sun, Dec 30, 2018 at 11:24 PM Cameron Simpson <cs at cskk.id.au> wrote:

> On 30Dec2018 21:14, Christian Seberino <cseberino at gmail.com> wrote:
> >What is simplest way to make both those
> >prints give same values?  Any slicker way
> >than an if statement?
>
> If your post had an attachment, be aware that the python-list list drops
> all attachments - it is a text only list. Please paste your code
> directly into your messages so that others can easily see it.
>
> Thanks,
> Cameron Simpson <cs at cskk.id.au>
>
-- 
_______________________________________

Christian Seberino, Ph.D.
Phone: (936) 235-1139
Email: cseberino at gmail.com
_______________________________________


From sandeep.bayi6 at gmail.com  Mon Dec 31 01:10:49 2018
From: sandeep.bayi6 at gmail.com (sandeep.bayi6 at gmail.com)
Date: Sun, 30 Dec 2018 22:10:49 -0800 (PST)
Subject: Facing an Error after migrating from python 3.4.1 to python 3.6.6
 ( Failed to import the site module  )
In-Reply-To: <lxbm55c0lf.fsf@cochabamba.vanoostrum.org>
References: <79ca23c8-2a95-40eb-b3a8-0887a5803765@googlegroups.com>
 <lxbm55c0lf.fsf@cochabamba.vanoostrum.org>
Message-ID: <a744a2e9-04d8-4102-8b99-603e86e05e27@googlegroups.com>

On Friday, December 28, 2018 at 5:41:52 PM UTC+5:30, Piet van Oostrum wrote:
> sandeep.bayi6 at gmail.com writes:
> 
> > ``````````````````````````````````````````````````````````````````````````` 
> > Error code: 
> > -------------------------------------------------------------------------- 
> >
> >
> > Traceback (most recent call last): 
> >   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\site.py", line 73, in <module 
> >     import os 
> >   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\os.py", line 652, in <module> 
> >     from _collections_abc import MutableMapping 
> >   File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\_collections_abc.py", line 58 
> >     async def _coro(): pass 
> >             ^ 
> > SyntaxError: invalid syntax 
> > Failed to import the site module 
> >
> > ```````````````````````````````````````````````````````````````````````````` 
> > After migrating from python 3.4.1 to python 3.6.6 
> > while Executing my project, I'm facing this issue and not able to
> > resolve it. Can i get any solution for this issue?
> 
> Could it be that your PYTHONPATH environment variable is set to a directory in Python 3.4.1?
> 
> -- 
> Piet van Oostrum <piet-l at vanoostrum.org>
> WWW: http://piet.vanoostrum.org/
> PGP key: [8DAE142BE17999C4]

Thankyou, but not worked.

Then i tried by removing the python 3.6.6 variables from PYTHONPATH and now , it is working fine for me 


From sandeep.bayi6 at gmail.com  Mon Dec 31 01:33:00 2018
From: sandeep.bayi6 at gmail.com (sandeep.bayi6 at gmail.com)
Date: Sun, 30 Dec 2018 22:33:00 -0800 (PST)
Subject: Error while calling a subprocess and execute another script from one
 py file
Message-ID: <7f579751-1b3c-4870-966e-906ab7d6c186@googlegroups.com>

Hi all,

==========================================================================
Error code :
--------------------------------------------------------------------------

Traceback (most recent call last):
  File "E:\ocius_tjb\run.py", line 163, in <module>
    subprocess.check_call(['C:/Python34/python.exe', logxml_parser, '-i', arg1, '-o', arg2], cwd=cur_work_dir)
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 286, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 267, in call
    with Popen(*popenargs, **kwargs) as p:
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 971, in _execute_child
    args = list2cmdline(args)
  File "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 461, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'module' is not iterable

--------------------------------------------------------------------------

> I'm currently facing this error while execute another py_script from one script.
> and i need to execute the program from python 3.6.6 and in  subprocess from python 3.4.1

Can i get a solution for this problem im facing?


From ian.g.kelly at gmail.com  Mon Dec 31 02:34:35 2018
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 31 Dec 2018 00:34:35 -0700
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <20181231052439.GA2726@cskk.homeip.net>
References: <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>
 <20181231052439.GA2726@cskk.homeip.net>
Message-ID: <CALwzidnnWZ_UitX9H2iorUr9x7339o5zA1d-yv6S1T-ajiaW8A@mail.gmail.com>

On Sun, Dec 30, 2018 at 10:27 PM Cameron Simpson <cs at cskk.id.au> wrote:
>
> On 30Dec2018 21:14, Christian Seberino <cseberino at gmail.com> wrote:
> >What is simplest way to make both those
> >prints give same values?  Any slicker way
> >than an if statement?
>
> If your post had an attachment, be aware that the python-list list drops
> all attachments - it is a text only list. Please paste your code
> directly into your messages so that others can easily see it.

The Google group has an initial post in this thread that didn't make it
through to the mailing list for whatever reason. For posterity, here
it is:

> Why are the following two similar prints slightly different and how fix?
>
> >>> x = 0x739ad43ed636
>
> >>> print(x + (-x) // 2048)
> 127046758190683
>
> >>> print(x - x // 2048)
> 127046758190684
>
> I'm working in an area where such deviations matter.  It would nice to understand what is happening.
>
> Any help greatly appreciated.
>
> cs


From rosuav at gmail.com  Mon Dec 31 02:47:24 2018
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 31 Dec 2018 18:47:24 +1100
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <CALwzidnnWZ_UitX9H2iorUr9x7339o5zA1d-yv6S1T-ajiaW8A@mail.gmail.com>
References: <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>
 <20181231052439.GA2726@cskk.homeip.net>
 <CALwzidnnWZ_UitX9H2iorUr9x7339o5zA1d-yv6S1T-ajiaW8A@mail.gmail.com>
Message-ID: <CAPTjJmramFetAS5q0aPk0zvYS00gjTG0ZAnEbsmF2uNW6r3nRQ@mail.gmail.com>

On Mon, Dec 31, 2018 at 6:36 PM Ian Kelly <ian.g.kelly at gmail.com> wrote:
>
> The Google group has an initial post in this thread that didn't make it
> through to the mailing list for whatever reason. For posterity, here
> it is:

Thanks Ian.

> > Why are the following two similar prints slightly different and how fix?
> >
> > >>> x = 0x739ad43ed636
> >
> > >>> print(x + (-x) // 2048)
> > 127046758190683
> >
> > >>> print(x - x // 2048)
> > 127046758190684
> >
> > I'm working in an area where such deviations matter.  It would nice to understand what is happening.

This is a distinctly different order of operations. Remember from
grade school that division is done before addition and subtraction;
the first one will calculate (-x)//2048 and then add that onto x, and
the second will calculate x//2048 and then subtract that from x. As
has already been pointed out, Python's // operator is *floor
division*, meaning that it will always round down, whether the number
is positive or negative.

ChrisA


From ian.g.kelly at gmail.com  Mon Dec 31 02:49:53 2018
From: ian.g.kelly at gmail.com (Ian Kelly)
Date: Mon, 31 Dec 2018 00:49:53 -0700
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>
References: <9618b456-e9c0-486d-977b-a45e7f694f14@googlegroups.com>
 <890c5fcc-b5a9-4ba3-a8d5-959ad3f19e5b@googlegroups.com>
 <CAPTjJmqX4E8DeeaPjW+k1vG9nJNmznYxKoJRxW7gfMsPuJ8hRw@mail.gmail.com>
 <mailman.1.1546225155.4816.python-list@python.org>
 <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>
Message-ID: <CALwzidkbvCaNpcN8tHLwJ7BQ12zpZmeHn-Km0Z78KNN-c-UtfA@mail.gmail.com>

On Sun, Dec 30, 2018 at 10:18 PM Christian Seberino <cseberino at gmail.com> wrote:
>
> What is simplest way to make both those
> prints give same values?  Any slicker way
> than an if statement?

Stack Overflow has a few suggestions:
https://stackoverflow.com/questions/19919387/in-python-what-is-a-good-way-to-round-towards-zero-in-integer-division


From cs at cskk.id.au  Mon Dec 31 04:48:02 2018
From: cs at cskk.id.au (Cameron Simpson)
Date: Mon, 31 Dec 2018 20:48:02 +1100
Subject: Question about slight deviations when using integer division
 with large integers.
In-Reply-To: <CAG5-5iKJQ=CixrK9ce4fbSeV1uXvwJvEaCHDRE-wB+eOHHOisw@mail.gmail.com>
References: <CAG5-5iKJQ=CixrK9ce4fbSeV1uXvwJvEaCHDRE-wB+eOHHOisw@mail.gmail.com>
Message-ID: <20181231094802.GA12400@cskk.homeip.net>

On 30Dec2018 23:33, Christian Seberino <cseberino at gmail.com> wrote:
>Thanks.  I didn?t post new code.  I was just referring back to original
>post.

I think Ian looked up the first post on Google Groups, where your code 
was evident. The message was incomplete when it got here (the mailing 
list); I don't know why.

>I need to duplicate the exact behavior of Java?s BigIntegers.
>I?m guessing difference between Java and Python is that Java 
>BigIntegers do
>not switch to floor for negatives.
>Possible to tweak rounding of Python to be like Java?

Directly but hacking the int type? Probably not. What you probably want 
to do is to make a JavaBigInteger Python class which subclasses int, and 
change its operator implementation. Eg (totally untested):

    class JavaBigInteger(int):

        def __floordiv__(self, other):
            sign = 1
            if self < 0:
                self = -self
                sign = -1
            if other < 0:
                other = - other
                sign *= -1
            result = self // other
            result *= sign
            return JavaBigOnteger(result)

i.e. convert both operands to positive values, divide, adjust the sign.

Some notes:

Changing __floordiv__ needs to have matching changes to __divmod__ and 
__mod__ because they're supposed to be consistent.

You probably also need to implement __truediv__, probably a lot like 
__floordiv__. There are also __rtruediv__ and __rfloordiv__ and __rmod__ 
and __rdivmod__.

You need to turn whatever integers you're working with into 
JavaBigIntegers to make them use the new operator methods, and those 
methods should themselves return JavaBigIntegers to keep the behaviour:

    # x and y are ints
    x = 11
    y = -2
    print(x // y)

    # you already have x and y from somewhere, eg a file
    # they are ints, make new JavaBigIntegers from them
    xj = JavaBigInteger(x)
    yj = JavaBigInteger(y)
    print(xj // yj)

Cheers,
Cameron Simpson <cs at cskk.id.au>


From p.f.moore at gmail.com  Mon Dec 31 06:12:06 2018
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 31 Dec 2018 11:12:06 +0000
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <CAG5-5iKJQ=CixrK9ce4fbSeV1uXvwJvEaCHDRE-wB+eOHHOisw@mail.gmail.com>
References: <c89dd5ca-c928-476c-be3b-8ffe61b9bcda@googlegroups.com>
 <20181231052439.GA2726@cskk.homeip.net>
 <CAG5-5iKJQ=CixrK9ce4fbSeV1uXvwJvEaCHDRE-wB+eOHHOisw@mail.gmail.com>
Message-ID: <CACac1F-MCB=01+XYxfaAzKqrDZRiKAt7R4wWFaapRJ3uXNhWVQ@mail.gmail.com>

On Mon, 31 Dec 2018 at 09:00, Christian Seberino <cseberino at gmail.com> wrote:
>
> Thanks.  I didn?t post new code.  I was just referring back to original
> post.  I need to duplicate the exact behavior of Java?s BigIntegers.
>
> I?m guessing difference between Java and Python is that Java BigIntegers do
> not switch to floor for negatives.

Presumably Java BigInteger division consistently rounds to zero,
whereas Python's // operator consistently rounds down.

> Possible to tweak rounding of Python to be like Java?

The correct answer is to have your developers understand the behaviour
of the language they are using, and not assume it's like another
language that they are more familiar with. But I appreciate that's not
always easy. So what you are looking for are ways to help your
developers avoid errors.

Python doesn't have any way to change the behaviour of the //
operator. I assume Java doesn't have a way to make BigInteger division
round down, either?

I doubt that using a custom-defined class in Python would help much -
developers would likely not use it, unless they understood the reason
for it (at which point, they wouldn't need it!). Code reviews,
focusing on the use of the // operator, might be an answer (hopefully
only needed short term until your developers understood the different
behaviours of Java and Python). Or maybe some form of coding
convention (all uses of // must be covered by unit tests that check
all combinations of signs of the 2 operands). Or maybe a function
java_style_divide(), that your conventions mandate must be used in
preference to //

Paul


From cseberino at gmail.com  Mon Dec 31 13:23:39 2018
From: cseberino at gmail.com (Christian Seberino)
Date: Mon, 31 Dec 2018 10:23:39 -0800 (PST)
Subject: Question about slight deviations when using integer division with
 large integers.
In-Reply-To: <87sgye58df.fsf@nightsong.com>
References: <9618b456-e9c0-486d-977b-a45e7f694f14@googlegroups.com>
 <87sgye58df.fsf@nightsong.com>
Message-ID: <ae13524e-220a-4067-b592-38196ec80667@googlegroups.com>

Thanks to all who helped.  As was previously pointed out, many other languages
use truncation rather than rounding for // division.

Getting the behavior you want may be as easy as replacing // with the int()
function....


>>> x = 9 ; y = 2

>>> x // y, -x // y, (-x) // y
(4, -5, -5)

>>> int(x / y), -int(x / y), int(-x / y)
(4, -4, -4)

Hope that helps.

Chris